Cooperative minimally invasive telesurgical system

ABSTRACT

Improved robotic surgical systems, devices, and methods include selectably associatable master/slave pairs, often having more manipulator arms than will be moved simultaneously by the two hands of a surgeon. Four manipulator arms can support an image capture device, a left hand tissue manipulation tool, a right hand tissue manipulation tool, and a fourth surgical instrument, particularly for stabilizing, retracting, tool change, or other functions benefiting from intermittent movement. The four or more arms may sequentially be controlled by left and right master input control devices. The fourth arm may be used to support another image capture device, and control of some or all of the arms may be transferred back-and-forth between the operator and an assistant. Two or more robotic systems each having master controls and slave manipulators may be coupled to enable cooperative surgery between two or more operators.

CROSS-REFERENCES TO RELATED APPLICATIONS

[0001] The present application is a continuation-in-part of and claimsthe benefit of priority from application Ser. No. 09/399,457, filed Sep.17, 1999 for a “Cooperative Minimally Invasive Telesurgical System”;application Ser. No. 09/374,643, filed Aug. 16, 1999 for a “CooperativeMinimally Invasive Telesurgical System”; and also claims the benefit ofpriority from Provisional Application Serial No. 60/116,891, filed Jan.22, 1999, for “Dynamic Association of Master and Slave in a MinimallyInvasive Telesurgical System”; Provisional Application Serial No.60/116,842, filed Jan. 22, 1999, for “Repositioning and Reorientation ofMaster/Slave Relationship in Minimally Invasive Telesurgery”; andProvisional Application Serial No. 60/109,359, filed Nov. 20, 1998, for“Apparatus and Method for Tracking and Controlling Cardiac Motion DuringCardiac Surgery Without Cardioplegia”; the full disclosures of which areincorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] The present application is generally directed to medical devices,systems, and methods. In a particular embodiment, the invention providestelesurgical robotic systems and methods that flexibly and selectablycouple input devices to robotic manipulator arms during surgery.

[0003] Advances in minimally invasive surgical technology coulddramatically increase the number of surgeries performed in a minimallyinvasive manner. Minimally invasive medical techniques are aimed atreducing the amount of extraneous tissue that is damaged duringdiagnostic or surgical procedures, thereby reducing patient recoverytime, discomfort, and deleterious side effects. The average length of ahospital stay for a standard surgery may also be shortened significantlyusing minimally invasive surgical techniques. Thus, an increasedadoption of minimally invasive techniques could save millions ofhospital days, and millions of dollars annually in hospital residencycosts alone. Patient recovery times, patient discomfort, surgical sideeffects, and time away from work may also be reduced with minimallyinvasive surgery.

[0004] The most common form of minimally invasive surgery may beendoscopy. Probably the most common form of endoscopy is laparoscopy,which is minimally invasive inspection and surgery inside the abdominalcavity. In standard laparoscopic surgery, a patient's abdomen isinsufflated with gas, and cannula sleeves are passed through small(approximately ½ inch or less) incisions to provide entry ports forlaparoscopic surgical instruments. The laparoscopic surgical instrumentsgenerally include a laparoscope (for viewing the surgical field) andworking tools. The working tools are similar to those used inconventional (open) surgery, except that the working end or end effectorof each tool is separated from its handle by an extension tube. As usedherein, the term “end effector” means the actual working part of thesurgical instrument and can include clamps, graspers, scissors,staplers, image capture lenses, and needle holders, for example. Toperform surgical procedures, the surgeon passes these working tools orinstruments through the cannula sleeves to an internal surgical site andmanipulates them from outside the abdomen. The surgeon monitors theprocedure by means of a monitor that displays an image of the surgicalsite taken from the laparoscope. Similar endoscopic techniques areemployed in, e.g., arthroscopy, retroperitoneoscopy, pelviscopy,nepbroscopy, cystoscopy, cisternoscopy, sinoscopy, hysteroscopy,urethroscopy, and the like.

[0005] There are many disadvantages relating to current minimallyinvasive surgical (MIS) technology. For example, existing MISinstruments deny the surgeon the flexibility of tool placement found inopen surgery. Most current laparoscopic tools have rigid shafts, so thatit can be difficult to approach the worksite through the small incision.Additionally, the length and construction of many endoscopic instrumentsreduces the surgeon's ability to feel forces exerted by tissues andorgans on the end effector of the associated tool. The lack of dexterityand sensitivity of endoscopic tools is a major impediment to theexpansion of minimally invasive surgery.

[0006] Minimally invasive telesurgical robotic systems are beingdeveloped to increase a surgeon's dexterity when working within aninternal surgical site, as well as to allow a surgeon to operate on apatient from a remote location. In a telesurgery system, the surgeon isoften provided with an image of the surgical site at a computerworkstation. While viewing a three-dimensional image of the surgicalsite on a suitable viewer or display, the surgeon performs the surgicalprocedures on the patient by manipulating master input or controldevices of the workstation. The master controls the motion of aservomechanically operated surgical instrument. During the surgicalprocedure, the telesurgical system can provide mechanical actuation andcontrol of a variety of surgical instruments or tools having endeffectors such as, e.g., tissue graspers, needle drivers, or the like,that perform various functions for the surgeon, e.g., holding or drivinga needle, grasping a blood vessel, or dissecting tissue, or the like, inresponse to manipulation of the master control devices.

[0007] While the proposed robotic surgery systems offer significantpotential to increase the number of procedures that can be performed ina minimally invasive manner, still further improvements are desirable.In particular, previous proposals for robotic surgery often emphasizedirect replacement of the mechanical connection between the handles andend effectors of known minimally invasive surgical tools with a roboticservomechanism. Work in connection with the present invention suggeststhat integration of robotic capabilities into the operating theater canbenefit from significant changes to this one-to-one replacement model.Realization of the full potential of robotically assisted surgery mayinstead benefit from significant revisions to the interactions and rolesof team members, as compared to the roles performed by surgical teammembers during open and known minimally invasive surgical procedures.

[0008] In light of the above, it would be beneficial to provide improvedrobotic surgical devices, systems, and methods for performing roboticsurgery. It would be beneficial if these improved techniques enhancedthe overall capabilities of telesurgery by recognizing, accommodating,and facilitating the new roles that may be performed by the team membersof a robotic surgical team. It would further be beneficial if theseimprovements facilitated complex robotic surgeries such as coronaryartery bypass grafting, particularly while minimizing the total numberof personnel (and hence the expense) involved in these roboticprocedures. It would be best if these benefits could be provided whileenhancing the overall control over the surgical instruments and safetyof the surgical procedure, while avoiding excessive complexity andredundancy in the robotic system. Some or all of these advantages areprovided by the invention described hereinbelow.

SUMMARY OF THE INVENTION

[0009] The present invention provides improved robotic surgical systems,devices, and methods. The robotic surgical systems of the presentinvention will often include a plurality of input devices and/or aplurality of robotic manipulator arms for moving surgical instruments. Aprocessor will often selectably couple a selected input device to aselected manipulator arm, and allows modification of the operativeassociation so that that same input device can be coupled to a differentmanipulator arm, and/or so that that same manipulator arm can becontrolled by a different input device. This selective coupling, forexample, allows the controller to properly assign left and rightsurgical end effectors to left and right input devices for use by anoperator viewing the procedure via an image capture device. When theimage capture devices moves, the operative associations can be revised.In some embodiments, the image capture device may be removed from one ofthe manipulator arms and instead mounted to another of the manipulatorarms, with the left and right input devices reassigned so as to avoid anawkward surgical environment for the system operator sitting at a mastercontrol station.

[0010] The systems of the present invention will often include moremanipulator arms than will be moved simultaneously by a single surgeon.In addition to an imaging arm (movably supporting an image capturedevice) and two manipulator arms (holding selectably designated “left”and “right” surgical tools for manipulation by left and right hands ofthe system operator, e.g.), one or more additional manipulator arms willoften be provided to position associated surgical instrument(s). Atleast one of the additional manipulator arms may be maintained in astationary configuration to stabilize or retract tissue while theoperator moves left and right input devices with his or her left andright hands to manipulate tissues with the associated surgical tools.The one or more additional arms may also be used to support anotherimage capture device, often a second endoscope to view the internalsurgical site from an alternative vantage point. Additional arms mayoptionally provide one or more additional surgical tools formanipulating tissue or aiding in the performance of a procedure at asurgical site. To manipulate this additional surgical instrument, anassistant input device may optionally be provided so that an assistant(such as an assisting surgeon at another workstation, a surgical nurseat the patient's side, or the like) can position the additional roboticarm(s). Regardless of the presence or absence of such an assistant inputdevice, the robotic surgical systems of the present invention willpreferably allow the system operator to selectively associate the rightand/or left input devices of the surgical workstation with any of aplurality of surgical instruments.

[0011] In a first aspect, the invention provides a robotic surgicalsystem comprising a first input device manipulatable by a hand of anoperator. A first robotic arm assembly includes a first manipulator armfor moving a first surgical instrument. A second robotic arm assemblyincludes a second manipulator arm for moving a second surgicalinstrument. A control system couples the first input device to the firstand second robotic arm assemblies. The control system permits selectiveoperative association of the first input device with the first roboticarm assembly, and also permits selective operative association of thefirst input device with the second robotic arm assembly.

[0012] Typically, the control system will have a plurality of selectablemodes, with manipulation of the first input device effectingcorresponding movement of the first surgical instrument in one mode, andwith the same manipulation of the first input device effectingcorresponding movement of the second surgical instrument in a secondmode. In some embodiments, a second input device may be used to effectcorresponding movement of the second surgical instrument when thecontrol system is in the first mode, and effect corresponding movementof the first surgical instrument when the control system is in thesecond mode, allowing swapping control of the surgical instrumentsbetween the input devices. This is particularly useful when the systemoperator is controlling the first and second input devices using leftand right hands with reference to an image of an internal surgical site,as it allows the system operator to switch tools when the image capturedevice providing the image moves to what would otherwise be an awkwardposition.

[0013] In another aspect, the invention provides a robotic surgicalsystem comprising a plurality of input devices and a plurality ofmanipulator arms, each manipulator arm having an instrument holder. Aplurality of surgical instruments are mountable to the instrumentholders, the surgical instruments including an image capture device anda tool having a surgical end effector for treating tissue. A controlsystem couples the input devices with the manipulator arms. The controlsystem selectably associates each input device with a manipulator arm.

[0014] In some embodiments, the image capture device may be removed fromone manipulator arm and mounted to an alternative manipulator arm, oftenwith the control system being reconfigurable so that the input devicesare operatively associatable with manipulator arms holding tools fortreating tissue. A particularly advantageous control system is providedwhich allows this flexible pairing of input devices and manipulatorarms.

[0015] In a specific aspect, the invention provides a minimally invasiverobotic surgical system comprising two input devices and at least twomedical instrument robotic arm assemblies. One of the input devices isoperatively associated with one of the robotic arm assemblies to causemovement of the robotic arm assembly in response to inputs on the inputdevice. The other input device is operatively associated with another ofthe robotic arm assemblies to cause movement of that other robotic armassembly in response to inputs on that other input device. A controlsystem couples the input devices with the robotic arm assemblies. Thecontrol system enables selective swapping so as to cause the inputdevice to be operatively associated with the robotic arm assembly whichwas operatively associated with the other input device, and to cause theother input device to be operatively associated with the robotic armassembly which was operatively associated with the input device. Relatedsystems allow selective operative association between at least one ofthe input devices and an image capture robotic arm assembly to permitthe position of an image capture device to be changed using the at leastone input device.

[0016] In a method aspect, the invention provides a robotic surgicalmethod comprising robotically moving a first surgical instrument using afirst manipulator arm by manipulating a first input device with a hand.A control system is reconfigured by entering a command. The controlsystem couples the first input device with the first manipulator arm,and also with a second manipulator arm. A second surgical instrument ismoved robotically using the second manipulator arm by manipulating theinput device with the hand after the reconfiguring step.

[0017] In a related method, a robotic surgical method comprisesrobotically moving a surgical instrument using a manipulator arm bymanipulating a first input device with a first hand. A control system isreconfigured by entering a command. The control system couples the firstinput device, and also a second input device, with the manipulator arm.The surgical instrument is robotically moved using the manipulator armby manipulating the second input device with a second hand after thereconfiguring step.

[0018] In another aspect, the invention provides a robotic surgicalsystem comprising an imaging system transmitting an image from an imagecapture device to a display. First, second, and third manipulator armseach support an associated surgical instrument. A master controller isdisposed adjacent the display. The master has a first input devicemanipulatable by a first hand of an operator, and a second input devicemanipulatable by a second hand of the operator. A processor operativelycouples each input device of the master to an associated manipulator armso that movement of the input device effects movement of the associatedsurgical instrument.

[0019] In some embodiments, the surgical instrument associated with thethird arm will comprise another image capture device. Where two imagecapture devices are included in the system, the processor willpreferably transmit arm movement command signals to the arms accordingto different coordinate system transformations depending on which imagecapture device is providing the image currently shown on the display.This allows the processor to correlate between a direction of movementof the input device and the movement of the surgical instrument whenswitching between two different endoscopes having different fields ofview.

[0020] Preferably, at least the third robotic instrument arm should beconfigurable to maintain a stationary configuration under somecircumstances, with the arm in the stationary configuration inhibitingmovement of the associated surgical instrument despite movement of theinput devices. Such a stationary configuration is particularly usefulwhen the surgical instrument mounted on the third arm comprises astabilizer (such as a coronary tissue stabilizer used for beating heartsurgery) or a retractor (for example, to retract tissue to expose adesired area of the cystic duct to the surgeon during cholecystectomy).The third arm will often comprise a linkage having a series of joints,and a brake system coupled to the joints to releasably inhibitarticulation of the linkage. The third arm linkage will preferably alsohave a repositionable configuration allowing manual articulation of thearm, and at least some of the arms will often remain stationary and/orbe repositionable in response to a signal. In addition to comprising arobotic manipulator arm having a driven configurations, the third armmay alternatively comprise a simple passive linkage with a brake systembut without actuators.

[0021] Preferably, the surgical system will include four or more roboticmanipulator arms. One of the additional arms may support an imagecapture device of the imaging system. To allow the operator toselectively manipulate all of these surgical instruments, including theimage capture device, the processor will have an operation mode in whichthe first arm moves its associated surgical instrument at the surgicalsite in response to movement to the first input device, while the secondarm moves its associated surgical instrument in response to movement ofthe second input device. In response to an arm selection signal from atleast one arm selector input coupled to the processor, the processor canselectively change operating modes by decoupling the first arm from thefirst input device, and instead operatively couple the first inputdevice with the second arm, the third arm, or the fourth arm. Theprocessor will maintain some (or ideally all) of the decoupled arms inthe stationary configuration under some circumstances, although adecoupled arm could be controlled to move in a repetitive or automatedmanner until recoupled to the surgeon's input device. An example of suchautomated motion of a decoupled robotic arm includes motion tracking ofa beating heart.

[0022] The robotic surgical system will often include an assistant inputdevice. The processor can selectively associate one or more of the armswith the assistant input device, or with an input device of the surgeon,so that the one arm moves in response to movement of the selected inputdevice. Hence, the processor can “hand-off” control of at least one arm(and its associated surgical instrument) between surgeon and assistantinput stations. This may be useful when the assistant is removing andreplacing the surgical instrument from the arm or when the assistant ata second console is to perform a part of the surgical procedure such as“closing” a portion of the surgical site. This also allows the surgeonto selectively assign direct control over an instrument based on theskill required to use the instrument for a given task, thereby enhancingrobotic team effectiveness. The assistant may optionally be working atan assistant control station that can correlate direction of movement ofthe assistant input device with an image of the end effector shown in anassistant display. The “assistant” image might be either a differentimage from a second endoscope or a shared image from the primaryconsole's endoscope, thereby enabling the surgeon and assistant to viewthe same image of the surgical site and manipulate each of theirassigned and coupled instruments to cooperate in performing a surgicalprocedure. Alternatively, a simple video monitor and assistant inputdevice (such as a handle within the sterile field) may be provided forthe assistant, particularly for a patient-side assistant performing toolswaps, intermittent irrigation, or the like. In other embodiments, thesurgeon's ability to select from among three or more surgicalinstruments, and to selectively associate each instrument with each ofthe input devices, will reduce and/or eliminate the need for surgicalassistants. Decreasing the use of surgical assistants (and the time tocontinually direct and oversee the assistant's movements) cansignificantly decrease the time and expense of a surgical procedure.

[0023] Typically, for a system having four arms, for example, threesurgical instruments and the image capture device will be supported byfour manipulators each comprising an endoscopic instrument having anelongate shaft with a proximal end adjacent the manipulator and a distalend insertable into an internal surgical site within a patient bodythrough a minimally invasive surgical aperture. Preferably, themanipulators will support the surgical instrument so that the shaftsextend radially outwardly from a pattern of apertures (typicallyincisions) in a “spoked wheel”-type arrangement. In an exemplaryarrangement for performing cardiac surgery, the four shafts will besufficiently long to enter apertures on the right side of the patientbody, and to extend toward the left side of the patient body fortreating the heart. During at least a portion of the procedure, top andbottom apertures of the pattern may accommodate first and secondendoscopes to provide flexibility in the field of view, particularly forthe separate steps of harvesting a suitable supply artery and forming ofthe anastomosis during coronary artery bypass grafting (CABG).Additional ports can support additional surgical tools. When performinga CABG procedure without cardioplegia on a beating heart, at least oneof the apertures of the pattern will often accommodate a tissuestabilizer during at least a portion of the procedure.

[0024] In another aspect, the invention provides a robotic surgicalsystem comprising a plurality of manipulator arms and a plurality ofsurgical instruments, each instrument mounted to an associated arm. Amaster controller station has a master display for viewing by anoperator, a first input device for manipulation by a first hand of theoperator, and a second input device for manipulation by a second hand ofthe operator. An assistant input device for manipulation by a hand of anassistant is also provided. Preferably, a processor selectivelyoperatively couples the controllers to the arms to effect movement tothe surgical instruments in response to movement of the input devices.

[0025] In a method aspect, the invention provides a robotic surgicalmethod comprising robotically moving first and second surgicalinstruments at a surgical site with first and second robotic manipulatorarms by manipulating first and second input devices with first andsecond hands of the operator, respectively. The first input device isselectively associatable with a third manipulator arm, so that a thirdsurgical instrument can be robotically moved at the surgical site withthe third manipulator arm by manipulating the first input device withthe first hand of the operator.

[0026] In another method aspect, the invention provides a roboticsurgical method comprising robotically moving first and second surgicalinstruments at a surgical site with first and second robotic manipulatorarms by manipulating first and second input devices with first andsecond hands of the operator, respectively. A third surgical instrumentis positioned at the surgical site by articulating a linkage of a thirdmanipulator. Movement of the positioned third surgical instrument isimpeded at the surgical site by inhibiting movement of the thirdmanipulator.

[0027] In yet another method aspect, the invention provides a roboticsurgical method comprising robotically positioning a surgical instrumentat a surgical site with a manipulator arm by manipulating a first inputdevice with a hand of a first operator. The manipulator arm isselectively associated with a second input device, and the surgicalinstrument is robotically moved at the surgical site with themanipulator arm by manipulating the second input device with a hand of asecond operator.

[0028] In a still further method aspect, the invention provides arobotic surgical method comprising showing, on a display, a first viewof a surgical site from a first image capture device. A surgicalinstrument is robotically removed at the surgical site with amanipulator arm by manipulating an input device with a hand of anoperator while the operator views the first view of the surgical site onthe display. A second image capture device is selectively associatedwith the display, and the surgical instrument is robotically manipulatedat the surgical site with the arm by manipulating the input device whilethe operator views a second view of the surgical site from the secondimage capture device on the display.

[0029] In yet another method aspect, the invention provides a roboticcoronary artery bypass grafting method comprising introducing an imagecapture device into a chest cavity of a patient through an aperturedisposed along a right side of the patient. An image of a surgical siteadjacent the heart is displayed from the image capture device to anoperator. A surgical procedure is performed on the heart by moving asurgical instrument at the surgical site with at least one roboticmanipulator arm while the surgical instrument extends through anotheraperture disposed along the right side of the patient. Preferably,tissue manipulation during the surgical procedure will primarily beperformed by surgical tools extending through a pattern of aperturesalong the right side of the patient.

[0030] In another aspect, first and second robotic manipulators arecontrolled by a first operator with first and second controllers, andthird and fourth manipulators are controlled by a second operator withthird and fourth controllers. Both operators view an image of theoperating site captured by a single image capture device, and bothcooperate to perform a surgical procedure. Each operator may have aseparate dedicated viewing station for his use. Such cooperation mayinclude helping each other perform the procedure, passing objects backand forth between manipulators during the procedure, and passing controlof various of the manipulator arms. Both sets of manipulators may sharethe same reference point so that control may be transferred without lossof context.

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The invention will now be described, by way of example, and withreference to the accompanying diagrammatic drawings, in which:

[0032]FIG. 1 is a plan view of a telesurgical system and method forperforming a robotic minimally invasive surgical procedure;

[0033]FIG. 2 shows a three-dimensional view of a control station of atelesurgical system in accordance with the invention;

[0034] FIGS. 3A-C show three-dimensional views of an input deviceincluding an articulated arm and wrist to be mounted on the arm for usein the master control station of FIG. 2.

[0035]FIG. 4 shows a three-dimensional view of a cart of thetelesurgical system in accordance with the invention, the cart carryingthree robotically controlled manipulator arms, the movement of the armsbeing remotely controllable from the control station shown in FIG. 2;

[0036]FIGS. 5 and 5A show a side view and a perspective view,respectively, of a robotic arm and surgical instrument assembly inaccordance with the invention;

[0037]FIG. 6 shows a three-dimensional view of a surgical instrument ofthe invention;

[0038]FIG. 7 shows, at an enlarged scale, a wrist member and endeffector of the surgical instrument shown in FIG. 6, the wrist memberand end effector being movably mounted on a working end of a shaft ofthe surgical instrument;

[0039] FIGS. 8A-C illustrate alternative end effectors having surfacesfor stabilizing and/or retracting tissue.

[0040] FIGS. 9A-E illustrate another cart supporting a fourth roboticmanipulator arm in the telesurgical system of FIG. 1, and a bracket formounting a tool on the manipulator arm.

[0041]FIG. 10 shows a schematic three-dimensional drawing indicating thepositions of the end effectors relative to a viewing end of an endoscopeand the corresponding positions of master control input devices relativeto the eyes of an operator, typically a surgeon;

[0042]FIG. 11 shows a block diagram indicating one embodiment of acontrol system of the telesurgical system of the invention;

[0043] FIGS. 11A-D schematically illustrate block diagrams and datatransmission time lines of an exemplary controller for flexibly couplingmaster/slave pairs;

[0044]FIG. 12 shows a block diagram indicating the steps involved inmoving the position of one of the master controls relative to itsassociated end effector;

[0045]FIG. 13 shows a control diagram which indicates control stepsinvolved when the master control is moved relative to its associated endeffector as indicated in the block diagram of FIG. 12;

[0046]FIG. 14 shows a block diagram indicating the steps involved inmoving the position of one of the end effectors relative to itsassociated master control;

[0047]FIG. 15 shows a control diagram which indicates control stepsinvolved when the end effector is moved relative to its associatedmaster control as indicated in FIG. 14;

[0048]FIG. 16 shows a block diagram indicating the steps involved inmoving the position of a viewing end of an endoscope of the minimallyinvasive telesurgical system relative to the end effectors;

[0049]FIG. 17 shows a control diagram which indicates control stepsinvolved when the end of the endoscope is moved relative to the endeffectors as indicated in FIG. 16;

[0050]FIG. 18 shows a simplified block diagram indicating the stepsinvolved in realigning a master control device relative to itsassociated end effector;

[0051]FIG. 19 shows a block diagram indicating the steps involved inreconnecting a control loop between a master control device and itsassociated end effector;

[0052]FIG. 19A shows a block diagram indicating the steps involved insmoothly recoupling an input device with an end effector so as to avoidinadvertent sudden movements;

[0053]FIG. 20 shows a schematic diagram indicating an operator of theminimally invasive telesurgical system of the invention at the controlstation shown in FIG. 2;

[0054]FIG. 21 shows a schematic diagram of an image captured by anendoscope of the minimally invasive telesurgical system of the inventionas displayed on a viewer of the system;

[0055]FIG. 21A shows a schematic diagram of another image captured bythe endoscope of the minimally invasive telesurgical system of theinvention as displayed on the viewer of the system;

[0056]FIG. 22 shows another image captured by the endoscope of theminimally invasive telesurgical system of the invention as displayed onthe viewer of the system;

[0057]FIG. 22A shows a reference plane indicating a region in dashedlines which corresponds to an area where an automated determination ofwhich of two master control devices of the system is to be associatedwith which of two slaves of the system is not desired;

[0058]FIG. 22B shows a block diagram indicating steps involved indetermining an association between which of the two master controldevices is to be associated with which of the two slaves of the system;and

[0059]FIG. 22C shows a block diagram indicating steps involved when theassociation of one of the master control devices with a slave is to beswitched or swapped with the association of another master controldevice and slave;

[0060]FIGS. 23A and 23B illustrate a system and method for performingcoronary artery bypass grafting on a beating heart by selectivelyassociating robotic surgical instruments with master input controldevices;

[0061]FIG. 24 shows a block diagram indicating steps involved in amethod for handing-off control of robotic tools between a surgeon and anassistant;

[0062]FIG. 25A is a cross section, looking towards the head, through achest of a patient body, illustrating an endoscopic coronary arterybypass graft procedure in which the heart is treated by robotic toolsintroduced via a pattern of apertures on the right side of the patient;and

[0063]FIG. 25B illustrates an exemplary pattern of four apertures forfour robotic endoscopic instruments as used in the procedure of FIG.25A.

[0064]FIGS. 26 and 27 schematically illustrate alternative robotictelesurgical systems.

DESCRIPTION OF THE SPECIFIC EMBODIMENTS

[0065] This application is related to the following patents and patentapplications, the full disclosures of which are incorporated herein byreference: PCT International Application No. PCT/US98/19508, entitled“Robotic Apparatus”, filed on Sep. 18, 1998 (Attorney Docket No.17516-005510PC), U.S. Patent Application Serial No. 60/111,713, entitled“Surgical Robotic Tools, Data Architecture, and Use” (Attorney DocketNo. 17516-003200), filed on Dec. 8, 1998; U.S. Patent Application SerialNo. 60/111,711, entitled “Image Shifting for a Telerobotic System”(Attorney Docket No. 17516-002700), filed on Dec. 8, 1998; U.S. PatentApplication Serial No. 60/111,714, entitled “Stereo Viewer System forUse in Telerobotic System” (Attorney Docket No. 17516-001500), filed onDec. 8, 1998; U.S. Patent Application Serial No. 60/111,710, entitled“Master Having Redundant Degrees of Freedom” (Attorney Docket No.17516-001400), filed on Dec. 8, 1998, U.S. Patent Application No.60/116,891, entitled “Dynamic Association of Master and Slave in aMinimally Invasive Telesurgery System” (Attorney Docket No.17516-004700), filed on Jan. 22, 1999; and U.S. Pat. No. 5,808,665,entitled “Endoscopic Surgical Instrument and Method for Use,” issued onSep. 15, 1998.

[0066] As used herein, first and second objects (and/or their images)appear “substantially connected” if a direction of an incrementalpositional movement of the first object matches the direction of anincremental positional movement of the second object (often as seen inan image), regardless of scaling between the movements. Matchingdirections need not be exactly equal, as the objects (or the object andthe image) may be perceived as being connected if the angular deviationbetween the movements remains less than about ten degrees, preferablybeing less than about five degrees. Similarly, objects and/or images maybe perceived as being “substantially and orientationally connected” ifthey are substantially connected and if the direction of an incrementalorientational movement of the first object is matched by the directionof an incremental orientational movement of the second object (often asseen in an image displayed near the first object), regardless of scalingbetween the movements.

[0067] Additional levels of connectedness may, but need not, beprovided.

[0068] “Magnitude connection” indicates substantial connection and thatthe magnitude of orientational and/or positional movements of the firstobject and second object (typically as seen in an image) are directlyrelated. The magnitudes need not be equal, so that it is possible toaccommodate scaling and/or warping within a magnitude connected roboticsystem. Orientational magnitude connection will imply substantial andorientational connection as well as related orientational movementmagnitudes, while substantial and magnitude connection means substantialconnection with positional magnitudes being related.

[0069] As used herein, a first object appears absolutely positionallyconnected with an image of a second object if the objects aresubstantially connected and the position of the first object and theposition of the image of the second object appear at least tosubstantially match, i.e., to be at the same location, during movement.A first object appears absolutely orientationally connected with animage of the second object if they are substantially connected and theorientation of the first object and the second object at leastsubstantially match during movement.

[0070] Referring now to FIG. 1, a robotic surgical network 10 includes amaster control station 200 and a slave cart 300, along with any ofseveral other additional components to enhance the capabilities of therobotic devices to perform complex surgical procedures. An operator Operforms a minimally invasive surgical procedure at an internal surgicalsite within patient P using minimally invasive surgical instruments 100.Operator 0 works at master control station 200. Operator 0 views adisplay provided by the workstation and manipulates left and right inputdevices. The telesurgical system moves surgical instruments mounted onrobotic arms of slave cart 300 in response to movement of the inputdevices. As will be described in detail below, a selectably designated“left” instrument is associated with the left input device in the lefthand of operator 0, and a selectably designated “right” instrument isassociated with the right input device in the right hand of theoperator.

[0071] As described in more detail in co-pending U.S. patent applicationSer. No., 09/373,678 entitled “Camera Referenced Control In A MinimallyInvasive Surgical Apparatus” and filed Aug. 13, 1999, (Attorney DocketNo. 17516-002110), the full disclosure of which incorporated herein byreference, a processor of master controller 200 will preferablycoordinate movement of the input devices with the movement of theirassociated instruments so that the images of the surgical tools 100, asdisplayed to the operator, appear at least substantially connected tothe input devices in the hands of the operator. Further levels ofconnection will also often be provided to enhance the operator'sdexterity and ease of use of surgical instruments 100.

[0072] Introducing some of the other components of network 10, anauxiliary cart 300A can support one or more additional surgical tools100 for use during the procedure. One tool is shown here forillustrative purposes only. A first assistant A1 is seated at anassistant control station 200A, the first assistant typically directingmovements of one or more surgical instruments not actively beingmanipulated by operator O via master control station 200. A secondassistant A2 may be disposed adjacent patient P to assist in swappinginstruments 100 during the surgical procedure. Auxiliary cart 300A mayalso include one or more assistant input devices 12 (shown here as asimple joy stick) to allow second assistant A2 to selectively manipulatethe one or more surgical instruments while viewing the internal surgicalsite via an assistant display 14. Preferably, the first assistant A1seated at console 200A views the same image as surgeon seated at console200. Further preferably, both the instruments of cart 300 and the“assistant” instruments of cart 300A are controlled according to thesame camera reference point, such that both surgeon and assistant areable to be “immersed” into the image of the surgical field whenmanipulating any of the tools.

[0073] As will be described hereinbelow, master control station 200,assistant controller 200A, cart 300, auxiliary cart 300A, and assistantdisplay 14 (or subsets of these components) may allow complex surgeriesto be performed by selectively handing-off control of one or morerobotic arms between operator O and one or more assistants.Alternatively, operator O may actively control two surgical tools whilea third remains at a fixed position, for example, to stabilize and/orretract tissue, with the operator selectively operating the retractor orstabilizer only at designated times. In still further alternatives, asurgeon and an assistant can cooperate to conduct an operation withouteither passing control of instruments or being able to pass control ofthe instruments, with both instead manipulating his or her owninstruments during the surgery, as will be described below withreference to FIG. 26. Although FIG. 1 depicts two surgeon consolescontrolling the two cart structures, a preferred embodiment comprisesonly one console controlling four or more arms on two carts. The scopemay optionally be mounted on the auxiliary cart and three tissuemanipulator arms may be mounted on the main cart. Generally, the use ofrobotic systems having four or more arms will facilitate complex roboticsurgical procedures, including procedures that benefit from selectableendoscope viewing angles. Methods for using robotic network 10 will bedescribed in more detail following descriptions of the networkcomponents. While the network component connections are schematicallyillustrated in FIGS. 1, 26, and 27, it should be understood that morecomplex interconnections between the various network components may beprovided.

[0074] Component Descriptions

[0075] Referring to FIG. 2 of the drawings, the control station of aminimally invasive telesurgical system in accordance with the inventionis generally indicated by reference numeral 200. The control station 200includes a viewer or display 202 where an image of a surgical site isdisplayed in use. A support 204 is provided on which an operator,typically a surgeon, can rest his or her forearms while gripping twomaster controls (FIGS. 3A and 3B), one in each hand. The master controlsare positioned in a space 206 inwardly beyond the support 204. Whenusing control station 200, the surgeon typically sits in a chair infront of the control station 200, positions her eyes in front of theviewer 202, and grips the master controls, one in each hand, whileresting her forearms on the support 204.

[0076] An example of one of the master control input devices is shown inFIGS. 3A-C, and is generally indicated by reference numeral 210. Themaster control device generally comprises an articulate positioning arm210A supporting orientational gimbals 210B. Gimbals 210B (shown mostclearly in FIG. 3B) have a plurality of members or links 212 connectedtogether by joints 214, typically by rotational joints. The surgeongrips the master control 210 by positioning his or her thumb and indexfinger over a grip actuation handle, here in the form of a grip handleor pincher formation 216. The surgeon's thumb and index finger aretypically held on the pincher formation by straps (not shown) threadedthrough slots 218. To move the orientation of the end effector, thesurgeon simply moves the pincher formation 216 to the desired endeffector orientation relative to the image viewed at the viewer 202, andthe end effector orientation is caused to follow the orientation of thepincher formation. Appropriately positioned positional sensors, e.g.,encoders, or potentiometers, or the like, are coupled to each joint ofgimbals 210B, so as to enable joint positions of the master control tobe determined as also described in greater detail herein below.

[0077] Gimbals 210B are similarly repositioned by movement of pincherformation 216, and this positional movement is generally sensed byarticulation of input arm 210A as shown in FIG. 3A. Reference numerals1-3 indicate orientational degrees of freedom of gimbals 210B, whilenumeral 4 in FIGS. 3A and 3B indicates the joint with which the mastercontrol and the articulated arm are connected together. When connectedtogether, the master control 210 can also displace angularly about axis4.

[0078] The articulated arm 210A includes a plurality of links 220connected together at joints 222. Articulated arm 210A has appropriatelypositioned electric motors to provide for feedback as described ingreater detail below. Furthermore, appropriately positioned positionalsensors, e.g., encoders, or potentiometers, or the like, are positionedon the joints 222 so as to enable joint positions of the master controlto be determined as further described herein below. Axes A, B, and Cindicate the positional degrees of freedom of articulated arm 210A. Ingeneral, movement about joints of the master control 210B primarilyaccommodates and senses orientational movement of the end effector, andmovement about the joints of arm 210A primarily accommodates and sensestranslational movement of the end effector. The master control 210 isdescribed in greater detail in U.S. Provisional Patent ApplicationSerial No. 60/111,710, and in U.S. patent application Ser. No.09/398,507, filed concurrently herewith (Attorney Docket No.17516-001410), the full disclosures of which are incorporated herein byreference.

[0079] As described more fully in co-pending U.S. patent applicationSer. No. 09/373,678, the full disclosure of which is incorporated hereinby reference, the orientation of the viewer relative to the mastercontrol input devices will generally be compared with the position andorientation of the end effectors relative to a field of view of theimage capture device. The relative locations of the input devices can bederived from knowledge regarding the input device linkage jointconfigurations (as sensed by the joint sensors), the construction anddesign of the master controller structure, and in some cases,calibration measurements taken from a specific master control consolesystem after fabrication. Such calibration measurements may be stored ina non-volatile memory of the console.

[0080] In FIG. 4 of the drawings, the cart 300 is adapted to bepositioned close to a surgical platform in preparation for surgery, andcan then be caused to remain stationary until a surgical procedure hasbeen completed. The cart 300 typically has wheels or castors to renderit mobile. The control station 200 may optionally be positioned remotefrom the cart 300, but will often be in or adjacent the operating room.The cart 300 carries three robotic arm assemblies. One of the roboticarm assemblies, indicated by reference numeral 302, is arranged to holdan image capturing device 304, e.g., an endoscope, or the like. Each ofthe two other arm assemblies 310 is arranged to carry a surgicalinstrument 100. The robotic arms are supported by positioning linkages395, which can be manually positioned and then locked in place before(or re-positioned during) the procedure.

[0081] The positioning linkages or “set-up joints” are described inProvisional Application Serial No. 60/095,303, the full disclosure ofwhich is incorporated herein by reference. Preferably, the set-up jointsinclude joint sensors which transmit signals to the processor indicatingthe position of the remote center of rotation. It should be noted thatthe manipulator arm assemblies need not be supported by a single cart.Some or all of the manipulators may be mounted to a wall or ceiling ofan operating room, separate carts, or the like. Regardless of thespecific manipulator structures or their mounting arrangement, it isgenerally preferable to provide information to the processor regardingthe location of insertion/pivot points of the surgical instruments intothe patient body. The set-up joint linkages need not have joint drivesystems but will often include a joint brake system, as they will oftenhold the manipulators in a fixed position during some or all of asurgical procedure.

[0082] The endoscope 304 has a viewing end 306 at a remote end of anelongate shaft. The elongate shaft of endoscope 304 permits it to beinserted into an internal surgical site of a patient's body. Theendoscope 304 is operatively connected to the viewer 202 to display animage captured at its viewing end 306 on the viewer 202.

[0083] Each robotic arm 302, 310 can be operatively connected to one ormore of the master controls 210 so that the movement of instrumentsmounted on the robotic arms is controlled by manipulation of the mastercontrols. The instruments 100 carried on the robotic arm assemblies 310have end effectors, generally indicated at 102, which are mounted onwrist members, the wrists in turn being pivotally mounted on distal endsof elongate shafts of the instruments. It will be appreciated that theinstruments have elongate shafts to permit them to be inserted into aninternal surgical site of a patient's body. Movement of the endeffectors relative to the ends of the shafts of the instruments is alsocontrolled by the master controls.

[0084] In FIGS. 5 and 5A of the drawings, one of the robotic manipulatorarm assemblies 310 is shown in greater detail. Assembly 310 includes anarticulated robotic arm 312, and a surgical instrument, schematicallyand generally indicated by reference numeral 100, mounted thereon.

[0085]FIG. 6 indicates the general appearance of the surgical instrument100 in greater detail. The surgical instrument 100 includes an elongateshaft 104. The wrist-like mechanism, generally indicated by referencenumeral 106, is located at a working end of the shaft 104. A housing108, arranged to releasably couple the instrument 100 to the robotic arm312, is located at an opposed end of the shaft 104. In FIG. 6, and whenthe instrument 100 is coupled or mounted on the robotic arm 312, theshaft 104 extends along an axis indicated at 109.

[0086] Referring again to FIGS. 5 and 5A, the instrument 100 istypically releasably mounted on a carriage 314, which is driven totranslate along a linear guide formation 316 of the arm 312 in thedirection of arrows P. The robotic arm 312 is typically mounted on abase by means of a bracket or mounting plate 317, which is affixed tothe passively movable set-up joints 395. Set-up joints 395 are held in afixed configuration during manipulation of tissue by a set-up jointbrake system. The base may be defined by the mobile cart or trolley 300,which is retained in a stationary position during a surgical procedure.

[0087] The robotic arm 312 includes a cradle, generally indicated at318, an upper arm portion 320, a forearm portion 322 and the guideformation 316. The cradle 318 is pivotally mounted on the plate 317 ingimbaled fashion to permit rocking movement of the cradle in thedirection of arrows 326 as shown in FIG. 5, about a pivot axis 328. Theupper arm portion 320 includes link members 330, 332 and the forearmportion 322 includes link members 334, 336. The link members 330, 332are pivotally mounted on the cradle 318 and are pivotally connected tothe link members 334, 336. By use of this linkage, irrespective of themovement of the robotic arm 312, a pivot center 349 remains in the sameposition relative to plate 317 with which the arm 312 is mounted. Inuse, the pivot center 349 is positioned at an aperture or a port ofentry into a patient's body when an internal surgical procedure is to beperformed.

[0088] While this “remote” center of motion-type arrangement for roboticmanipulation is described in connection with the preferred embodimentsof this invention, the scope of the inventions disclosed herein is notso limited, encompassing other types of arrangements such as manipulatorarms having passive or natural centers of motion at the point ofinsertion into a patient body.

[0089] The robotic arm 312 provides three degrees of freedom of movementto the surgical instrument 100 when mounted thereon. These degrees offreedom of movement are firstly the gimbaled motion indicated by arrows326, pivoting movement as indicated by arrows 327 and the lineardisplacement in the direction of arrows P. These three degrees offreedom of movement are primarily coupled to translational degrees ofmovement of the end effector, although some rotational coupling may bepresent. Movement of the arm as indicated by arrows 326, 327 and P iscontrolled by appropriately positioned electrical motors which respondto inputs from an associated master control to drive the arm 312 to arequired position as dictated by movement of the master control.Appropriately positioned sensors, e.g., potentiometers, or the like, areprovided on the arm to determine joint positions as described in greaterdetail herein below.

[0090] Referring now to FIG. 7 of the drawings, the wrist mechanism 106will now be described in greater detail. In FIG. 7, the working end ofthe shaft 104 is indicated at 110. The wrist mechanism 106 includes awrist member 112. One end portion of the wrist member 112 is pivotallymounted in a clevis, generally indicated at 117, on the end 110 of theshaft 104 by means of a pivotal connection 114. The wrist member 112 canpivot in the direction of arrows 156 about the pivotal connection 114.

[0091] An end effector, generally indicated by reference numeral 102, ispivotally mounted on an opposed end of the wrist member 127. The endeffector 102 is in the form of, e.g., a clip applier for anchoring clipsduring a surgical procedure. Accordingly, the end effector 102 has twoelements 102.1, 102-2 together defining a jaw-like arrangement. It willbe appreciated that the end effector can be in the form of any surgicaltool having two members which pivot about a common pivotal axis, such asscissors, pliers for use as needle drivers, or the like. Instead, it caninclude a single working member, e.g., a scalpel, cautery electrode, orthe like. Alternative non-articulated tools may also be used, includingtools for aspiration and/or irrigation, endoscopes, or the like. When atool other than a clip applier is required during the surgicalprocedure, the tool 100 is simply removed from its associated arm andreplaced with an instrument bearing the required end effector, e.g., ascissors, or pliers, or the like.

[0092] The end effector 102 is pivotally mounted in a clevis, generallyindicated by reference numeral 119, on an opposed end of the wristmember 112, by means of a pivotal connection 160. Elements 102.1, 102.2are angularly displaceable about the pivotal connection 160 toward andaway from each other as indicated by arrows 162, 163. It will further beappreciated that the elements 102.1, 102.2 can be displaced angularlyabout the pivotal connection 160 to change the orientation of the endeffector 102 as a whole, relative to the wrist member 112. Thus, eachelement 102.1, 102.2 is angularly displaceable about the pivotalconnection 160 independently of the other, so that the end effector 102,as a whole, is angularly displaceable about the pivotal connection 160as indicated in dashed lines in FIG. 7. Furthermore, the shaft 104 isrotatably mounted on the housing 108 for rotation as indicated by thearrows 159. Thus, the end effector 102 has three degrees of freedom ofmovement relative to the arm 112 in addition to actuation of the endeffector, preferably namely, rotation about the axis 109 as indicated byarrows 159, angular displacement as a whole about the pivot 160 andangular displacement about the pivot 114 as indicated by arrows 156.Other wrist structures and combinations of joints also fall within thescope of the present inventions, however. For example, while thisarrangement and these resulting degrees of freedom of movement arepreferred, a wrist having fewer degrees of freedom of movement, such asa single distal articulating joint, or a wrist having othersingularities, may also be used, as desired.

[0093] The three degrees of freedom of movement of instrument 100 areprimarily coupled to orientational degrees of freedom of movement of theend effector. This is somewhat a simplification, as movement about thesethree axes will result in some change in position of the end effector.Similarly, movement about the above-described translational axes maycause some changes in orientation. It will be appreciated thatorientational movement of the end effector, like translational movement,is controlled by appropriately positioned electrical motors whichrespond to inputs from the associated master control to drive the endeffector 102 to a desired position as dictated by movement of the mastercontrol. Furthermore, appropriately positioned sensors, e.g., encoders,or potentiometers, or the like, are provided to determine jointpositions as described in greater detail herein below. In thisspecification the actuation or movement of the end effectors relative toeach other in the directions of arrows 62, 63 is not regarded as aseparate degree of freedom of movement.

[0094] Tissue stabilizer end effectors 120 a, b, and c, referred togenerally as tissue stabilizers 120, are illustrated in FIGS. 8A-C.Tissue stabilizers 120 may have one or two end effector elements 122that preferably are pivotally attached to the distal end of the shaft orwrist of a surgical instrument and are moveable with respect to oneanother, and that preferably comprise tissue-engaging surfaces 124. Thetissue-engaging surfaces optionally include protrusions, ridges, vacuumports, or other surfaces adapted so as to inhibit movement between theengaged tissue and the stabilizer, either through pressure applied tothe engaged tissue or vacuum applied to draw the tissue into an at leastpartially stabilized position, or a combination of both pressure andvacuum. The ideal tissue engaging surface will constrain and/or reducemotion of the engaged tissue in the two lateral (sometimes referred toas the X and Y) axes, along the tissue-engaging surface, and thestabilizer configuration and engagement with the tissue will at leastpartially decrease motion normal to the surface. Other configurationsfor traditional stabilizers are known to those of skill in the art, suchas the Octopus II of Medtronic, Inc. and various HeartPort, Inc. andCardioThoracic Systems stabilizers having multipronged and doughnutconfigurations. These manners of contacting tissue allow stabilizers 120to firmly engage a moving tissue such as a beating heart of a patientand reduce movement of the tissue adjacent the stabilizer.

[0095] To facilitate performing a procedure on the stabilized tissue, anopening 126 may be formed in an individual stabilizer element 122,and/or between independently moveable end effector elements. Asillustrated in FIG. 8B, stabilizer 120 b includes cooperating tissuegrasping surfaces 128 disposed between stabilizer end effector elements122. This allows the stabilizer to grasp tissues, providing a dualfunction robotic stabilizer/grasper tool. Stabilizer 120 b may be used,for example, as a grasper while harvesting and/or preparing an internalmammary artery (IMA) for a coronary artery bypass graft (CABG)procedure, and/or to hold the IMA during formation of the anastomosis onthe stabilized beating heart.

[0096] In general, tissue stabilizers 120 will have a sufficiently smallprofile, when aligned with shaft 104 of instrument 100, to allow thestabilizer to advance axially through a cannula. Similar (or modified)end effectors having high friction tissue-engaging surfaces may be usedas retractors to hold tissue clear of a surgeon's line of sight during aprocedure.

[0097] Referring now to FIG. 8C, and generally for the roboticendoscopic stabilizers disclosed herein, each stabilizer may comprise anirrigation port 125, the port preferably in fluid communication with alumen integrated into the shaft of the stabilizer tool. While anirrigation and/or aspiration capability is particularly beneficial whenincorporated into a stabilizer, such capabilities may also beincorporated into the shaft of any robotic surgical tool, as desired.The port system, comprising a lumen preferably situated inside the shaftof the stabilizer and extending out of an aperture or port in the distalportion of the shaft, may be used to perform a number of tasks during asurgical procedure (e.g., a beating heart procedure) in whichstabilization of tissue is desired. Those tasks may include removingundesired fluid from the surgical site (either through suction tooutside the patient's body), blowing the fluid into some other portionof the surgical site, and/or delivering fluid (such as spray humidifiedcarbon dioxide) to clear the surgical site of material (such as bodyfluids which might otherwise interfere with the surgeon's view).Preferably, at least the distal portion of the port system is flexibleto permit bending. The exemplary port structure will be malleable orplastically deformable enough that it will hold its position whenrepositioned.

[0098] To take advantage of the irrigation aspect of thismulti-functional stabilizer, the stabilizer is inserted with the distalexternal portion of the irrigation device preferably flush with theshaft of the stabilizer. After the stabilizer has reached the surgicalsite, the operator may reposition the irrigation port distal end withone of the other surgical manipulators by grasping the port structureand moving it to a desired location and/or orientation relative to shaft104, wrist 106, or end effector element 122 (depending on the structureto which the port is mounted). The device may remain in that locationfor the duration of the surgery, or may be moved around as desired. Inaddition to simply being moveable at the surgical site, the device alsomay be extendable from/retractable into the stabilizer shaft, so thatthe distal end can be moved towards or away from the surgical siteitself, as desired.

[0099] An example of a preferred auxiliary cart 300A is seen in moredetail in FIGS. 9A and B. Auxiliary cart 300A includes a simple linkage350 with sliding joints 352 which can be releasably held in a fixedconfiguration by latches 354. Linkage 350 supports an auxiliary remotecenter manipulator arm 302A having a structure similar to arm 302 usedto support the endoscope on cart 300. (See FIG. 4.) The linkagestructure of auxiliary arm 302A is described more fully in co-pendingU.S. Patent Application Serial No. 60/112,990, filed on Dec. 16, 1998,the full disclosure of which is incorporated herein by reference.Generally, auxiliary arm 302A effectively includes a parallel linkagemechanism providing a remote center of spherical rotation 349 at a fixedlocation relative to base 317, similar to that described above withreference to arm 312 in FIG. 5. Although this arm is described aspreferably being of different structure that other instrumentmanipulator arms also described herein, it should be understood that aother manipulator arm can also be used either to support an endoscope orto serve as the fourth arm on the auxiliary cart 300A.

[0100] Sliding joints 352 and wheels 356 (which can also be releasablylocked in a fixed configuration by latches 354) allow remote center orfulcrum 349 to be positioned at an insertion point into a patient bodyusing translational movement along X, Y, and Z axes. Auxiliary arm 302Amay optionally be actively driven so as to translationally position ashaft of a surgical instrument within a patient body. Alternatively,auxiliary arm 302A may be used as a passive manipulator arm. Auxiliaryarm 302A (like all manipulator arms of the robotic network) preferablyincludes a repositioning configuration input device or button 358,ideally disposed on a manual positioning handle 360. When repositioningbutton 358 is depressed, the joints of auxiliary arm 302A move freely soas to pivot the arm about fulcrum 349 manually. Once actuator 358 isreleased, auxiliary arm 302A remains in a substantially fixedconfiguration. The arm will resist movement until repositioning button358 is again held down, or until the arm receives an actuation signalfrom an associated master control input device. Hence, auxiliary cart300A may be used to support a surgical instrument such as an endoscope,a stabilizer, a retractor, or the like, even if not actively drivenunder direction of an input device.

[0101] Manual repositioning of the supported surgical instrument willgenerally be performed by an assistant under the direction of a surgeonin charge of the surgical procedure. Typically, even when the set-upjoints 395, cart linkages 350, arms 302, 312, and/or other structures ofthe robotic system support the end effectors in a fixed configuration,the brake or motor drive systems inhibiting movement of the instrumentscan be safely overridden using manual force without damaging the roboticsystem. This allows repositioning and/or removal the instruments if afailure occurs. Preferably, the override force will be sufficient toinhibit inadvertent movement from accidental bumping, interferencebetween manipulators, and the like.

[0102] Auxiliary arm 302A and arm 302 used to support endoscope 304 neednot necessarily include a drive system for articulating a wrist and/orend effectors within the patient body, unless, e.g., a wrist is to beused in connection with a stabilizer to improve positioning of theparticular tissue to be stabilized. When auxiliary cart 300A is to beused to actively drive an articulated tool under the direction of anoperator O or assistant via a processor, arm 302 may optionally bereplaced by arm 312. Alternatively, where the auxiliary cart is to beused as a passive structure to hold an articulated surgical instrumentat a fixed position and configuration within a patient body, a manualtool articulation bracket 370 may be used to mount the tool 100 toauxiliary arm 302A. The manual tool bracket 370 is illustrated in FIGS.9C-9E.

[0103] As can be understood with references to FIGS. 9C and 9D, bracket370 comprises a plate 372 with sidewalls which fittingly receive housing108 of tool 100. Discs 374 have drive surfaces which drivingly engagethe drive system of tool 100 so as to rotate shaft 104 about its axis,articulate the end effector about the wrist, and move the first andsecond end effector elements, as described above.

[0104] As seen most clearly in FIG. 9E, the rotational position of discs374 can be changed by manually rotating adjustment knobs 376, which arerotationally coupled to the discs. Once the instrument 100 is in thedesired configuration, lock nuts 378 may be tightened against washers379 to rotationally affix knobs 376 and discs 374. In the exemplaryembodiment, bracket 372 comprises a polymer, while knobs 376 and nuts378 may be polymeric and/or metallic. Washer 379 may comprise a lowfriction polymer, ideally comprising a PTFE such as Teflon™, or thelike. While the disclosure herein shows a preferred embodiment formanual manipulation of a stabilizer by a surgical assistant, it shouldbe apparent that the stabilizer might just as easily be controlled froma remote robotic control console, from which the operator wouldmanipulate the stabilizer and any associated wrist in the same way asother instruments are controlled, as herein described.

[0105] Telesurgical Methods and Component Interactions

[0106] In use, the surgeon views the surgical site through the viewer202. The end effector 102 carried on each arm 312, 302, 302A is causedto perform movements and actions in response to movement and actioninputs of its associated master control. It will be appreciated thatduring a surgical procedure images of the end effectors are captured bythe endoscope together with the surgical site and are displayed on theviewer so that the surgeon sees the movements and actions of the endeffectors as he or she controls such movements and actions by means ofthe master control devices. The relationship between the end effectorsat the surgical site relative to the endoscope tip as viewed through theviewer and the position of the master controls in the hands of thesurgeon relative to the surgeon's eyes at the viewer provides anappearance of at least a substantial connection between the mastercontrols and the surgical instrument for the surgeon.

[0107] To provide the desired substantial connection between the endeffector images and the master controller input devices, the processorof master control station 200 and/or assistant control station 200A willgenerally map the internal surgical worksite viewed by the endoscopeonto the master controller work space in which the operator and/orassistant moves his or her hands. The position of the arms holding thesurgical tools relative to the arm holding the endoscope in use may beused to derive the desired coordinate transformations so as to providethe desired level of substantial connectedness, as more fully explainedin co-pending U.S. Patent Provisional Application Serial No. 60/128,160,previously incorporated herein by reference.

[0108] Where a tool is to be viewed through an endoscope, and the tooland endoscope are supported by independent support structures (forexample, when viewing a tool supported by arm 312 within the internalsurgical site via an endoscope supported by auxiliary cart 300A) it isparticularly beneficial to have a known orientation between the twoindependent support structures to allow the desired transformations tobe derived. This may be provided, for example, by ensuring that the basestructure of cart 300 is accurately parallel to the base structure ofauxiliary cart 300A. As positional transformations and modifications arerelatively straightforward when orientations are accurately aligned,this allows a processor to provide substantial connection despite theseparately mounted robotic network components.

[0109] The operation of telesurgical robotic network 10 will first beexplained with reference to interaction between master control station200 and cart 300. Many of the aspects of this interaction appear in theinteractions among the remaining network components.

[0110] Master-Slave Controller

[0111] In FIG. 10, the Cartesian space coordinate system is indicatedgenerally by reference numeral 902. The origin of the system isindicated at 904. The system 902 is shown at a position removed from theendoscope 304. In the minimally invasive telesurgical system of theinvention, and for purposes of identifying positions in Cartesian space,the origin 904 is conveniently positioned at the viewing end 306. One ofthe axes, in this case the Z-Z axis, is coincident with the viewing axis307 of the endoscope. Accordingly, the X-X and Y-Y axes extend outwardlyin directions perpendicular to the viewing axis 307.

[0112] It will be appreciated that in the case of angular displacementof the endoscope to vary the orientation of the displayed image asdescribed above, the reference plane defined by the X-X and Y-Y axis isangularly displaced together with the endoscope.

[0113] As mentioned earlier, when the surgical instruments are mountedon the arms 112, a fulcrum 349 or pivot point is defined for each armassembly 310. Furthermore, as also already mentioned, each fulcrum 349is positioned at a port of entry into the patient's body. Thus,movements of the end effectors at the surgical site is caused by angulardisplacements about each fulcrum 349. As described above, the locationof the fulcrums may be sensed using joint sensors of the set-up joints,or using a variety of alternative position sensing systems.

[0114] When the remote center or fulcrum positions relative to theviewing end 306 of the endoscope 304 are determined, the coordinates inthe X-X and Y-Y plane of the Cartesian coordinate system 902 aredetermined. It will be appreciated that these (X,Y) coordinates of eachfulcrum 349 can vary depending on the chosen entry ports to the surgicalsite. The location of these entry ports can vary depending on thesurgical procedure to be performed. It will further be appreciated thatthe (X,Y) coordinates of each fulcrum 349 can readily be determined withreference to the coordinate system 902 by means of the position sensorsat the various pivot points on each robotic arm 112 since the endoscope304 and the arms 310 are mounted on the same cart 300. Naturally, theendoscope arm 302 is also provided with appropriately positionedpositional sensors. Thus, to determine the (X,Y) coordinates of eachfulcrum 349, relative to the coordinate system 902, the position of thecoordinate system 902 can be determined relative to any arbitrary pointin space by means of the positional sensors on the endoscope arm 302 andthe positions of each fulcrum relative to the same arbitrary point canreadily be determined by means of the positional sensors on each roboticarm 112. The positions of each fulcrum 349 relative to the coordinatesystem 902 can then be determined by means of routine calculation.

[0115] With reference to FIG. 11, a control system defining a controlloop which links master control inputs to end effector outputs, and viceversa for feedback, is schematically indicated by reference numeral 400.Master control inputs and corresponding end effector outputs areindicated by arrows AB and end effector inputs and corresponding mastercontrol outputs in the case of feedback is indicated by arrows BA.

[0116] In this specification, for the sake of clarity, positions sensedby the encoders on the master which relate to joint positions arereferred to as “joint space” positions. Similarly, for the sensors onthe joints of the robotic arm and the wrist mechanism, positionsdetermined by these sensors are also referred to as “joint space”positions. The robotic arm and wrist mechanism will be referred to asthe slave in the description which follows. Furthermore, references topositions and positioned signals may include orientation, location,and/or their associated signals. Similarly, forces and force signals maygenerally include both force and torque in their associated signals.

[0117] For ease of explanation, the system 400 will be described from aninitial condition in which the master is at an initial position and theslave is at a corresponding initial position. However, in use, the slavetracks master position in a continuous manner.

[0118] Referring to the control system 400, the master is moved from theinitial position to a new position corresponding to a desired positionof the end effector as viewed by the surgeon in the image displayed onthe viewer 202. Master control movements are input by a surgeon at 402,as indicated by arrow AB1 by applying a force to the master control at404 to cause the master control to move from its initial position to thenew position.

[0119] As the master is moved, signals e_(m) from the encoders on themaster is input to a master input controller at 406 as indicated byarrow AB2. At the master input controller 406, the signals e_(m) areconverted to a joint space position θ_(m) corresponding to the newposition of the master. The joint space position θ_(m) is then input toa master kinematics converter 408 as indicated by arrow AB3. At 408 thejoint position θ_(m) is transformed into an equivalent Cartesian spaceposition x_(m). This is optionally performed by a kinematic algorithmincluding a Jacobian transformation matrix, inverse Jacobian (J-¹), orthe like. The equivalent Cartesian space position x_(m) is then input toa bilateral controller at 410 as indicated by arrow AB4.

[0120] Position comparison and force calculation may, in general, beperformed using a forward kinematics algorithm which may include aJacobian matrix. Forward kinematics algorithm generally makes use of areference location, which is typically selected as the location of thesurgeon's eyes. Appropriate calibration or appropriately placed sensorson console 200 can provide this reference information. Additionally, theforward kinematics algorithm will generally make use of informationconcerning the lengths and angular offsets of the linkage of the masterinput device 210. More specifically, the Cartesian position x_(m)represents the distance of the input handle from, and the orientation ofthe input handle relative to, the location of the surgeon's eyes. Hence,x_(m) is input into bilateral controller 410 as indicated by AB4.

[0121] In a process similar to the calculations described above, theslave location is also generally observed using sensors of the slavesystem. In the exemplary embodiment, the encoder signal e_(s) are readfrom the slave joint sensors at 416 as indicated by BA2, and are thenconverted to joint space at step 414. As indicated by BA3, the jointspace position of the slave is also subjected to a forward kinematicsalgorithm at step 412. Here, the forward kinematics algorithm ispreferably provided with the referenced location of tip 306 of endoscope304. Additionally, through the use of sensors, design specifications,and/or appropriate calibration, this kinematics algorithm incorporatesinformation regarding the lengths, offsets, angles, etc., describing thelinkage structure of patient cart 300, set-up joints 395, and roboticmanipulator arms 310, so that the slave Cartesian position x_(s)transferred at BA4 is measured and/or defined relative to the endoscopetip.

[0122] At bilateral controller 410, the new position of the master x_(m)in Cartesian space relative to the surgeon's eyes is compared with theinitial position x_(s) of the instrument tip in Cartesian space relativeto the camera tip. This relationship is depicted in FIG. 10 showing thetriangle connecting the surgeon's eye and the master controllers in thehands of the surgeon, as well as the triangle coupling camera tip 306and the end effectors of tools 104. Advantageously, the comparison ofthese relative relationships occurring in controller 410 can account fordifferences in scale between the master controller space in which theinput device is moved as compared with the surgical workspace in whichthe end effectors move. Similarly, the comparison may account forpossible fixed offsets, should the initial master and slave positionsnot correspond.

[0123] At 410, the new position x_(m) of the master in Cartesian spaceis compared with the initial position of the slave, also in Cartesianspace. It will be appreciated that the positions of the master and slavein Cartesian space are continually updated in a memory. Thus, at 410,the initial position of the slave in Cartesian space is downloaded fromthe memory so as to compare it with the new position of the master inCartesian space. Thus, the initial position of the slave in Cartesianspace was derived from the joint space position of the slave when boththe master and the slave were at their initial positions. It willfurther be appreciated that, at 410, and where the position of themaster in Cartesian space conforms with a corresponding position of theslave in Cartesian space, no positional deviation results from thecomparison at 410. In such a case no signals are sent from 410 to causemovement of the slave or the master.

[0124] Since the master has moved to a new position, a comparison of itscorresponding position x_(m) in Cartesian space with the Cartesian spaceposition of the slave corresponding to its initial position, yields apositional deviation. From this positional deviation in Cartesian space,a force f_(s) in Cartesian space is computed at 410 which is necessaryto move the slave position in Cartesian space to a new positioncorresponding to the new position of the master x_(m) in Cartesianspace. This computation is typically performed using a proportionalintegral derivative (P.I.D.) controller. This force f_(s) is then inputto a slave kinematics converter 412 as indicated by arrow AB5.Equivalent joint torques τ_(s) are computed in the slave kinematicsmodule, typically using a Jacobian transpose method. This is optionallyperformed by a Jacobian Transpose (J^(T)) controller.

[0125] The torques τ_(s) are then input to a slave output converter at414 as indicated by arrow AB6. At 414 currents is are computed. Thesecurrents is are then forwarded to the electrical motors on the slave at416 as indicated by arrow AB7. The slave is then caused to be driven tothe new position x_(e) which corresponds to the new position into whichthe master has been moved.

[0126] The control steps involved in the control system 400 as explainedabove are typically carried out at about 1300 cycles per second orfaster. It will be appreciated that although reference is made to aninitial position and new position of the master, these positions aretypically incremental stages of a master control movement. Thus, theslave is continually tracking incremental new positions of the master.

[0127] The control system 400 makes provision for force feedback. Thus,should the slave, typically the end effector, be subjected to anenvironmental force f_(e) at the surgical site, e.g., in the case wherethe end effector pushes against tissue, or the like, such a force is fedback to the master control. Accordingly, when the slave is trackingmovement of the master as described above and the slave pushes againstan object at the surgical site resulting in an equal pushing forceagainst the slave, which urges the slave to move to another position,similar steps as described above take place.

[0128] The surgical environment is indicated at 418 in FIG. 11. In thecase where an environmental force f_(e) is applied on the slave, such aforce f_(e) causes displacement of the end effector. This displacementis sensed by the encoders on the slave 416 which generate signals e_(s).Such signals e_(s) are input to the slave input converter 414 asindicated by arrow BA2. At the slave input 414 a position θ_(s) in jointspace is determined resulting from the encoder signals e_(s). The jointspace position θ_(s) is then input to the slave kinematics converter at412 and as indicated by arrow BA3. At 412 a Cartesian space positionx_(s) corresponding to the joint space position θ_(s) is computed andinput to the bilateral controller at 410 as indicated by arrow BA4. TheCartesian space position x_(s) is compared with a Cartesian spaceposition x_(m) of the master and a positional deviation in Cartesianspace is computed together with a force f_(m) required to move themaster into a position in Cartesian space which corresponds with theslave position x_(s) in Cartesian space. The force f_(m) is then inputto the master kinematics converter at 408 as indicated by arrow BA5.

[0129] From the f_(m) input, desired torque values τ_(m) are determinedat 408. This is typically performed by a Jacobian Transpose (J^(T))controller. The torque values are then input to the master outputconverter at 406 as indicated by arrow BA6. At 406, master electricmotor currents i_(m) are determined from the torque values τ_(m) and areforwarded to the master at 404 and as indicated by arrow BA7 to causethe motors to drive the master to a position corresponding to the slaveposition.

[0130] Although the feedback has been described with respect to a newposition desired by the master to track the slave, it will beappreciated that the surgeon is gripping the master so that the masterdoes not necessarily move. The surgeon however feels a force resultingfrom feedback Torques on the master which he counters because he isholding onto the master.

[0131] The discussion above relating to the control system 400 providesa brief explanation of one type of control system which can be employed.It will be appreciated that instead of using a Jacobian Transposecontroller, an Inverse Jacobian Controller arrangement can be used. Whenusing an inversed Jacobian controller, bilateral controller 410 mayoutput a Cartesian slave position command x_(sd) at AB5 to thekinematics module 412, with the Cartesian slave position commandindicating the desired position of the slave. Kinematics algorithmmodule 412 may then use, for example, an inverse Jacobian algorithm todetermine a desired joint space position θ_(sd) which can be comparedagainst the initial joint space position of the slave θ_(s). From thiscomparison, joint torques may be generated to compensate for anypositioning errors, with the joint torques passed via AB6 to the slaveinput/output module 414 as described above.

[0132] It should also be noted that control system 400 may coupleactuation of the master handle (in the exemplary embodiment, variationof the gripping angle defined between grip members 218 as shown in FIG.3B) to articulation of the end effector (in the exemplary embodiment,opening and closing the end effector jaws by varying the end effectorangle between end effector elements 102.1, 102.2 as illustrated in FIG.7) in the matter described above, by including the master grip input andthe end effector jaw actuation in the joint and Cartesian positioneffectors, equivalent torque vectors, and the like, in the calculationswhich have been described.

[0133] It should be understood that additional controllers or controllermodules may be active, for example, to provide friction compensation,gravity compensation, active driving of redundant joint linkage systemsso as to avoid singularities, and the like. These additional controllersmay apply currents to the joint drive systems of the master and slaves.The additional functions of these added controllers may remain even whenthe master/slave control loop is interrupted, so that termination of themaster/slave relationship does not necessarily mean that no torques areapplied.

[0134] An exemplary controller block diagram and data flow to flexiblycouple pairs of master controllers with manipulator arms are shown inFIGS. 11A-11D. As described above, the operator 402 manipulatesmanipulators 404, here inputting actuation forces against both the leftand right master manipulators f_(h) (L, R). Similarly, both left andright positions of the master input devices will also be accommodated bythe control system, as will forces and positions of four or more slavemanipulator arms f_(e) (1, 2, 3, and 4), x_(e) (1, 2, 3, and 4). Similarleft, right, and slave notations apply throughout FIGS. 11A-11D.

[0135] The encoder increments from each joint of the master inputdevices 404 and the slave manipulators 416 are all input into aservocontrol input pre-processor SCI. In some or all of the joints ofthe master or slave structures, this information may be provided in analternative format, such as with an analogue signal (optionallyproviding absolute position indication) from a Hall effect transducer, apotentiometer, or the like.

[0136] Where at least some of the signals transmitted from master inputdevices 404 or slave manipulators 416 comprise encoder increments,pre-processor SCI may include one or more accumulators 1002 asillustrated in FIG. 1l B. Positive and/or negative encoder incrementsare counted between servocycle transfer requests 1004, which areprovided from a servo timing generator STG are accumulated in a firstregister 1006. After receipt of transfer request 1004, the accumulatedencoder increments from throughout the servocycle are transferred tosecond register 1008.

[0137] As schematically illustrated in FIG. 11C, the transfer request ispreferably offset from an encoder increment clock so as to avoidinadvertent encoder reading errors during servocycle data transfer. Inother words, to avoid losing encoder increments during data transfer, anasynchronous transfer request/encoder increment sample rate ispreferably provided, as illustrated in FIG. 11C. The sample rate willoften be higher than the rate at which the encoder can produceincrements, and the accumulators will generally hold incrementalposition information for all encoder-equipped freely moveable joints ofthe input and slave manipulators over a servocycle, the servocyclepreferably having a frequency of over 900 Hz, more preferably having afrequency of 1,000 Hz or more, often having a frequency of at leastabout 1,200 Hz, and ideally having a frequency of about 1,300 Hz ormore.

[0138] Preferably, an accumulator 1002 will be included in pre-processorSCI for each encoder of the master input devices 404 and slavemanipulators 416. Each encoder accumulator will preferably accommodateat least a 12-bit joint position signal, and in many cases willaccommodate a 14-bit joint position signal. Where analogue positionsignals are provided, they will typically be converted to digitalsignals at or before storage in the pre-processor SCI, with as many as48 joint signals or more being provided in the exemplary pre-processor.

[0139] Referring now to FIGS. 11A and 11D, and first concentrating ontransmission to and from a first bilateral controller CE1 during aservocycle, joint positional information e_(m), e_(s) for a particularmaster input device 404/slave manipulator 416 pair is retrieved inresponse to a servointerrupt signal 1010 from the servo timing generatorSTG. The control processor CTP may transform these joint positionsignals to the desired coordinate reference frame, or may alternativelytransfer this information in joint space on to the bilateral controllerCE1 for conversion to the desired reference frame. Regardless, theposition is preferably transmitted from the control processor CTP to thebilateral controller CE1 using a direct memory access DMA controller orother high-speed data transmission system.

[0140] Once the positional information has been transferred from thecontrol processor CTP to controller CE1 at DMA interrupt 1012 (see FIG.11D), the controller processes the positional information, comparing theend effector positions in the surgical workspace with the input devicepositions (including both location and orientation) in the mastercontroller workspace.

[0141] As more fully explained in co-pending U.S. patent applicationSer. No. 09/373,678, filed Aug. 13, 1999, the full disclosure of whichis incorporated herein by reference, the surgical and controller seworkspaces may be scaled and positioned relative to each other asdesired, often using positional information provided by the sensors ofthe set-up joints, and incorporating calibration and/or assemblyinformation of the master control console so as to identify the locationand/or orientation of the master input device relative to the viewer. Ingeneral, as the structure supporting the image capture device and endeffectors on the slave side are known, and as the location of the viewerrelative to the master input device can be calculated from similarknowledge regarding the lengths of the master input lengths, the mastercontroller joint angles, and the like, an appropriate coordinationtransformation may be derived so as to mathematically couple the masterspace of the master control workstation and the slave space in thesurgical environment. The information on both the master and slavelinkages in structure may be based on a model of these linkage andsupport structures, on design specifications for the linkage and supportstructures, and/or on measurements of individual linkages, which may bestored in a non-volatile memory of the slave and/or master control, suchas by burning calibration information into a memory of the appropriatestructure.

[0142] As illustrated in FIG. 11D, much of a servocycle time is used bythe controller CE1 to calculate appropriate high-level instructions forthe master and slave systems. The results of these calculations aretransferred to control processor CTP via yet another DMA interrupt 1014.These high-level commands, typically in the form of desired forces to beapplied on the master and slave f_(m),f_(s) in a suitable referenceframe such as a Cartesian coordinate system are converted by the controlprocessor CTP to desired motor current signals, which are directed tothe appropriate motors by postprocessor SCO.

[0143] While the pre- and post-processors, timing generator, controlprocessor, and controllers are illustrated schematically in FIG. 11A asseparate blocks, it should be understood that some or all of thesefunctional components may be combined, for example, on a singleprocessor board, or that multiple processor boards may be used with thefunctions of one or more of these components being separated on toseparate processors.

[0144] As can be understood with reference to FIGS. 11A and 11D, whilethe first controller CE1 is processing the position and otherinformation associated with the first master/slave pair, the pre- andpost-processors and control processor are processing and transferringdata for use by the second and third controllers CE2, CE3. Hence, theindividual controllers have asynchronous input and output times. Itshould be understood that more than three controllers may be providedfor additional master/slave pairs. In the exemplary embodimentillustrated in FIG. 11A, for example, the first and second controllersCE1 and CE2 might be dedicated to left and right hand inputs from thesurgeon, while the third controller CE3 may be used to move theendoscope using the left and/or right input device, or any other desiredinput system.

[0145] In the embodiment of FIG. 11A, servo timing generator STGincludes a memory storing the master/slave pair assignments 1016. Thesepair assignments are communicated to the pre- and post-processors SCI,SCO, so that the information transferred to and from the controlprocessor CTP is appropriate for the controller, and so that thecommands from the appropriate controller are properly understood andtransmitted to the drive system for the appropriate joints. Reallocationof the master/slave pair assignments is transmitted to the timinggenerator STG from the control processor CTP, and is then communicatedfrom the timing generator to the pre-and postprocessors during anintermittent initialization phase, which may also be used to set upappropriate processor time intervals. Alternatively, the time intervalsmay be fixed.

[0146] As should be understood by those of skill in the art, theflexible master/slave pairing controller of FIG. 11A is still asimplification, and an appropriate controller will include a number ofadditional systems. For example, it is highly beneficial to includefault-checking software to ensure that all encoders or other jointsensors are read during each servocycle, and that the drive systems ofeach driven joint of the master and slave are written to during eachservocycle. If the fault-check is not successfully completed, the systemmay be shut down. Similarly, the control system may check for changes inpair assignments, for example, during data transfer to and/or from thecamera controller. Similarly, pair assignments may be reviewed duringand/or after a tool change, during a left/right tool swap, when handingoff tools between two different master controllers, when the systemoperator requests a transfer, or the like.

[0147] It should be noted that the control system of FIGS. 11A-11D mayaccommodate flexible tool mountings on the various manipulators. Asdescribed above, the first and second controllers CE1, CE2 may be usedto manipulate tools for treating tissue, while the third controller CE3is dedicated to tool movements using inputs from both master inputdevices. In general surgical procedures, it may desirable to remove theendoscope or other image capture device from a particular manipulatorand instead mount it on a manipulator which was initially used tosupport a treatment tool. By appropriate commands sent via the controlprocessor CTP to the servo timing generator STG, the pair assignmentsfor the three controllers may be revised to reflect this change withoutotherwise altering the system operator's control over the system.

[0148] During pair re-assignment, appropriate data sets and/ortransformations reflecting the kinematics of the master/slave pairs, therelationship of the image capture device with the end effectors, and thelike, may be transmitted to the controller. To facilitate swapping theimage capture device from one manipulator to another, it may bebeneficial to maintain a common manipulator structure throughout thesystem, so that each manipulator includes drive motors for articulatingtools, endoscope image transfer connectors, and the like. Ideally,mounting of a particular tool on a manipulator will automaticallytransmit signals identifying the tool to the control system, asdescribed in co-pending U.S. Patent Application Serial No. 60/111,719,filed on Dec. 8, 1998, (Attorney Docket No. 17516-003210) entitled“Surgical Robotic Tools, Data Architecture, and Use.” This facilitateschanging of tools during a surgical procedure.

[0149] A variety of adaptations of the exemplary control system will beobvious to those of skill in the art. For example, while the exemplaryembodiment includes a single master bus and a single slave bus, one orboth of these individual busses may be replaced with a plurality ofbusses, or they may be combined into a single bus. Similarly, while theexemplary servocycle time for an individual control pair is preferablyabout 1,000 msec or less, and ideally about 750 msec or less, the use ofhigher speed processing equipment may provide servocycle times which aresignificantly faster.

[0150] The master/slave interaction between master control station 200and cart 300 is generally maintained while the operator O is activelymanipulating tissues with surgical instruments associated with his orher left and right hands. During the course of a surgical procedure,this master/slave interaction will be interrupted and/or modified for avariety of reasons. The following sections describes selectedinterruptions of the master/slave control interaction, and are usefulfor understanding how similar interruptions and reconfigurations of thetelesurgical robotic network may be provided to enhance the capabilitiesof the overall robotic system. The exemplary interruptions include“clutching” (repositioning of a master control relative to a slave),repositioning of an endoscope, and a left-right tool swap (in which atool previously associated with a master control input device in a righthand of a surgeon is instead associated with an input device in a lefthand of the surgeon, and vice versa.) It should be understood that avariety of additional interruptions may occur, including during removaland replacement of a tool, during manual repositioning of a tool, andthe like.

[0151] Clutching

[0152] In the course of performing a surgical procedure, the surgeon maywish to translationally reposition one or both of the master controlsrelative to the position or positions of a corresponding end effector oreffectors as displayed in the image. The surgeon's dexterity isgenerally enhanced by maintaining an ergonomic orientational alignmentbetween the input device and the image of the end effector. The surgeonmay reposition the master relative to the end effector by simplyinterrupting the control loop and re-establishing the control loop inthe desired position, but this can leave the end effector in an awkwardorientation, so that the surgeon repeatedly opens the control loop toreorient the end effectors for each translational repositioning.Advantageously, the ergonomic rotational alignment between input devicesand the images of the end effectors can be preserved after the mastercontrol or controls have been repositioned by a modified clutchingprocedure, which will now be described with reference to FIGS. 12 and13.

[0153] Referring to FIG. 12, a block diagram indicating therepositioning of one of the master controls is indicated generally byreference numeral 450 and will now be described. It will be appreciatedthat both master controls can be re-positioned simultaneously. However,for ease of description, the repositioning of a single master controlwill be described. To reposition the master control relative to itsassociated slave, the surgeon causes the control loop 400 linking mastercontrol movement with corresponding slave movement to be interrupted.This is accomplished by activation by the surgeon of a suitable inputdevice, labeled “Depress Master Clutch Button” at 452 in FIG. 12. It hasbeen found that such a suitable input device can advantageously be inthe form of a foot pedal as indicated at 208 in FIG. 2. It will beappreciated that any suitable input can be provided such as voicecontrol input, a finger button, or the like. It is advantageous toprovide an input device which does not require the surgeon to remove hisor her hands from the master controls so as to preserve continuity ofmaster control operation. Thus, the input device can be incorporated onthe master control device itself instead of having a foot pedal.

[0154] Once the input has been activated, e.g., by depressing the footpedal, the control loop 400 between master and slave is interrupted. Theslave is then locked in its position, in other words in the position inwhich it was at immediately before the foot pedal was depressed.

[0155] As can be described with reference to FIG. 11, upon depression ofthe foot pedal, the link 410 in the control system 400 between masterand slave is interrupted. The position in joint space of the slaveimmediately before depression of the foot pedal is recorded in a memoryof a slave joint controller indicated at 420 in dashed lines. Should aforce then be applied to the slave to cause it to displace to a newjoint position, the encoders on the slave relay signals to 414 where anew joint space position for the slave is computed and forwarded to theslave joint controller 420 as indicated by arrow BA9. This new jointspace position is compared with the joint space position in the memory,and joint space deviations are determined. From this joint spacedeviation, torques are computed to return the slave to the jointposition as recorded in the memory. These torques are relayed to 414 asindicated by arrow AB9 where corresponding electric motor currents aredetermined which are forwarded to the slave motors to cause it torestore its joint space position. Thus, the slave position is servolocked.

[0156] Referring again to FIG. 11, upon depression of the foot pedal at452, the translational movement of the master is caused to float whileits orientation is locked, as indicated at 454 in FIG. 12. This step isachieved by a master Cartesian controller with memory as indicated at422 in FIG. 11. The functioning of the master Cartesian controller withmemory will now be described with reference to FIG. 13.

[0157] Upon activation of the foot pedal and repositioning of themaster, the joint space position input of the master control asindicated by θ_(a) is converted from joint space to Cartesian space at406. From this conversion, a Cartesian space position x_(a) of themaster is obtained. The position in Cartesian space of the masterimmediately before activation of the foot pedal is recorded in a memoryat 424 and is indicated by x_(d). The current position x_(a) of themaster as it moves to its new position is compared with the recordedposition x_(d) at 456 to obtain error signals, which correspond topositional deviations of current master position in Cartesian space whencompared with the recorded position x_(d) in Cartesian space. Thesedeviations or errors are input to a feedback controller at 426 todetermine a feedback force to return the master to a positioncorresponding to the recorded position x_(d). The components of thefeedback force which corresponds to translational movement are zeroed at428. Thus, translational feedback force components are zeroed and onlyorientational force components are forwarded from 428. The orientationalforce components are then converted to corresponding torques at 408,which are then input to 406 (in FIG. 11) to determine currents forfeeding to the electric motors on the master to cause its orientation tobe urged to remain in a condition corresponding to the orientationdetermined by x_(d). It will be appreciated that the orientation at theposition x_(d)corresponds to the orientation of the slave since theslave continuously tracks the master and the positions were recorded inmemory at the same time. Since the translational forces were zeroed, thetranslational movement of the master is caused to float enabling thesurgeon to translate the master to a new, desired position. Suchtranslational floating may alternatively be provided by a variety ofother methods. For example, the translational gains of controller 426may be set to zero. In some embodiments, the translational elements ofmemory 424 may be continually reset to be equal to the input valuesx_(a), so that the difference between the measured position and thestored position is zero. It should also be understood that despite thezeroing of the translational terms, additional controller functions suchas friction compensation, gravity compensation, or the like, may remainunaltered.

[0158] Referring again to FIG. 12 of the drawings, when the mastercontrols have been moved to their desired position the foot pedal isreleased. Upon release, the translational deviations relating to the newposition of the master control relative to its associated slave isincorporated into 410 to define a new Cartesian space position at whichthe slave position corresponds to the master position. In particular,the translational derivations may be incorporated in the fixed offsetsdescribed above, preferably using the algorithm described herein toavoid inadvertent sudden movements or forces.

[0159] Since the orientation of the end effector was held at the sameposition, and since the master orientation was caused to remain in acorresponding orientation, realignment of the end effector and master isnormally not necessary. Re-connection of master and slave takes placeupon release of the foot pedal as indicated at 456. The reconnectionwill now be described with reference to FIG. 19.

[0160] Referring to FIG. 19, a block diagram illustrating the stepsinvolved in re-connecting the control system 400 between the master andthe slave is generally indicated by reference numeral 470.

[0161] The first step involving in re-connecting control between themaster and the slave, and as indicated at 472, is to determine whetheror not the master orientation is sufficiently close to the slaveorientation. It will be appreciated that it could happen that duringrepositioning of the master as described above, the surgeon could beurging the pincher formation on the master away from its orientationallyaligned position relative to the slave. Should re-connection of controlbetween master and slave then occur, it could result in reactive motionby the slave resulting from the urging force applied by the surgeon onthe pincher formation. This reactive motion by the slave could causeunnecessary damage to organs, or tissue, or the like, at the surgicalsite and should be avoided. Accordingly, at 472 the orientation ofmaster and slave is compared. If the orientation of the master does notcoincide with the orientation of the slave or does not fall within anacceptable orientational deviation, re-connection of control betweenmaster and slave will not be enabled. In such a case an appropriatemessage is typically displayed on the viewer indicating to the surgeonthat a required corrective action is required to cause the orientationof the master to be within the acceptable deviational range relative tothe orientation of the slave. An example of such a message is oneindicating to the surgeon to relax his or her grip on the pincherformation. Simultaneously, the master alignment algorithm may beexecuted as described hereinbelow with reference to FIG. 18.

[0162] When the orientations of master and slave are sufficientlysimilar, the slave orientation is optionally snapped with the masterorientation in Cartesian space as indicated at 474. Once the orientationis snapped, the Jacobian Inverse controller on the slave is enabled asindicated at 476. Thereafter, the Cartesian force reflection commandsand gains are downloaded as indicated at 478.

[0163] As used herein, the snapping of the slave orientation to themaster orientation means that the orientational offsets in bilateralcontroller 410 are reset to zero, so that the master and slaveorientations begin tracking each other. In synchronization with thissnapping, the control system 410 is reconfigured to normal bilateralcontrol, preferably using a Jacobian inverse, as indicated at step 476.The appropriate commands and gains are downloaded as indicated at step478.

[0164] In many embodiments, rather than instantaneously snapping themaster to the slave, the orientational offsets in bilateral controller410 may alternatively be slowly and smoothly reduced to zero, therebyproviding a smoother transition between operating modes. This may beeffected by, for example, filtering the orientational offset values tozero.

[0165] In general, some and/or all transition of control system 400between operating configurations or modes, including those describedwith reference to steps 454 and 456 of the master repositioningalgorithm of FIG. 12, as well as a variety of similar steps describedhereinbelow, may include potentially substantially instantaneous changesin configuration or perimetric values of the control system. Forexample, interrupting or opening the loop of bilateral controller 410,enabling master Cartesian controller 422, resetting memory 424 or thecontroller gains in P.I.D. controller 426 might be performed bysubstantially instantaneously changing the perimetric values and/orconfigurations. Such instantaneous changes may be fundamentallydifferent than normal master/slave operation, where the computations arecontinually repeated using fixed perimetric values and operationalconfigurations, with only the sensor readings changing.

[0166] Where substantially instantaneous changes in perimetric valuesand/or configuration are imposed, it is possible that a sudden change inmotor currents may result, causing the system to jerk. Such inadvertentinstantaneous movements of the system may be transmitted to the surgeonor other system operator, and can be disconcerting and/or reduce theoverall feel of control the operator has over the system. Additionally,unexpected rapid movements of a surgical instrument at a surgical siteare preferably minimized and/or avoided. Hence, rather than effectingthese changes in perimetric values and/or configuration instantaneously,the changes will preferably be timed and executed in a manner so as toavoid significant instantaneous changes in the computed motor currentsapplied before, during, and after the change in configuration. Thissmooth change of perimetric values and/or controller configurations maybe provided by a “no-jerk” algorithm which will be described withreference to FIG. 19A.

[0167] The relevant control system mode transitions typically involve aconfiguration change, a change in a fixed memory value, or the like. Inparticular, bilateral controller 410 makes use of fixed offsets in itsmemory. Controllers 420, 422, and 560 also contain fixed commands intheir memories. The no-jerk algorithm, which generally decreases and/oreliminates rapid inadvertent movement of the master or slave, utilizesknown sensor readings, configuration information, and memory valuesimmediately before a control system operating mode transition. Byassuming that sensor readings will remain predictable, changing onlyslightly during the controller mode transition, the no-jerk algorithmcomputes desired memory reset values by also taking into account theknown end values or configuration, and by synchronizing the change invalues so as to promote smooth motor current changes during the modetransition. For some uses, the no-jerk algorithm my reduce or eliminatesudden changes in motor torques by using pre-transition (and optionallyfiltered) motor currents or joint torque values in place of or incombination with the pre-transition sensor configuration and memoryvalues as inputs.

[0168] Referring now to FIG. 19A, pre-transition configuration,perimetric values, and memory values are used, together with sampledpre-transition sensor values 702 to compute pre-transition joint torquesat step 704. Alternatively, these pre-transition joint torques may bedirectly observed, optionally with filtering, at step 706. Regardless,post-transition joint torque values are forced to match thepre-transition joint torque values at step 708. Meanwhile, using knownpost-transition configuration and perimetric values 710, thepost-transition effective feedback gains may be determined at step 712.These post-transition effective feedback gains may be inverted and usedtogether with the post-transition joint torques to calculate a desiredpost-transition error signal at step 714. The post-transition sensorvalues may be predicted at step 716. These post-transition sensor valuesmay be estimated by assuming that smooth sensor readings will beprovided, and knowing the time it takes to effect transition.

[0169] The desired post-transition error signal and predicted sensorvalues may be used to derive a desired post-transition command signal atstep 718.

[0170] Based on the known post-transition configuration, thepost-transition command signal will generally determine the desiredmemory or offset value through calculations performed at step 720. Thispost-transition memory or offset value is reset in synchronization withthe transition at step 722. Hence, once the desired mode transition isinput, information about the configuration of the system before andafter the change takes place allows smoothing of the transition.

[0171] Repositioning of one of the slaves relative to one of the masterswill now be described with reference to FIGS. 14 and 15. It is to beappreciated that both slaves can be repositioned relative to theirassociated masters simultaneously. However, for ease of explanation therepositioning of a single slave relative to its associated master willnow be described.

[0172] In FIG. 14 a block diagram indicating steps involved inrepositioning a slave relative to its associated master is generallyindicated by reference numeral 500. When it is desired to move the endeffector of a slave to a new position, a suitable input is activated tointerrupt the control loop 400 between the master and the slave. Such asuitable input can be in the form of a button on the robotic arm asindicated at 480 in FIG. 5A. Depressing such a button to interrupt thecontrol loop 400 is indicated by the term “Depress Slave Clutch Button”at 504 in FIG. 14. Once the button is depressed, the control betweenmaster and slave is interrupted to cause the translational movements ofthe slave to float while the orientation of the end effector is lockedas indicated at 502 in FIG. 14.

[0173] In general, when movements of one or more joints of a master orslave linkage are allowed to float, the floating joints may optionallystill have some forces imposed against the joint by their associatedjoint-drive systems. More specifically, as described more fully inco-pending U.S. patent application Ser. No. 09/287,513, the fulldisclosure of which is incorporated herein by reference, the controllermay impose actuation forces on the master and/or slave so as tocompensate for gravity, friction, or the like. These compensation forcesmay be maintained on the floating joint or joints even when the controllink for actuating the joint is otherwise open.

[0174] The step indicated at 502 will now be described in greater detailwith reference in particular to FIG. 15, and also with reference to FIG.11. When the button 480 is depressed, the position θ_(d) of the slave injoint space immediately before depression of the button is recorded in amemory of the slave joint controller 420, and as indicated at 460. Asthe slave is moved thereafter, its position in joint space indicated byθ_(a) is compared with θ_(d) at 462. As θ_(a) deviates from θ_(d) errorsignals corresponding to the positional deviation in joint space isdetermined at 462 and is passed to 464. At 464 required torques for theelectric motors on the slave are determined to cause the slave to returnto the θ_(d) position. The torques thus determined which relate totranslational torques of the slaves are zeroed at 466 to permit theslave translational movements to float. The torques corresponding toorientational movement are not zeroed. Thus, any environmental forces onthe end effector urging an orientational position change are fed back tothe end effector to cause it to retain its orientation. In this way theorientation of the end effector relative to the end of the instrumentshaft 104 is locked in position. Although the orientation of the endeffector does not change relative to the end of the shaft, it doeschange in position in Cartesian space as a result of translationalposition change. It should be understood that zeroing of the outer jointtorques at step 466 may be effected by a variety of methods, includingzeroing of the appropriate gains in P.I.D. controller 464, continuallyupdating the appropriate elements in memory 460 so as to compute a zeroerror signal at comparison 462, or the like.

[0175] It should be also be understood that a variety of additionaloperation configurations may be implemented which allow slavetransitional movements to float free of the master control. For example,slave transitional forces may be zeroed in Cartesian space (analogous tothe master clutching algorithm described with reference to FIGS. 12 and13). Alternatively, control system 400 and/or bilateral controller 410may be interrupted only for translational motions, locking the mastertranslational position and allowing the slave to float in transnationalposition, all while connecting the master orientation to the slaveorientation. Once the slave is at the desired position the button isreleased as indicated at 510 in FIG. 14.

[0176] When the button is released, the master orientation is re-alignedwith the slave orientation as indicated at 512. The re-aligning of theorientation of the master and slave is now described with reference toFIG. 18. The steps involved in such realignment are generally indicatedby reference numeral 550.

[0177] At 552 the slave position θ_(s) in joint space is read. Theposition θ_(s) is then converted to a position x_(s) in Cartesian spaceat 554 using slave forward kinematics. Thereafter at 556, the desiredorientation of the master is set to equal the slave orientation inCartesian space. Thus x_(m), the master orientational position inCartesian space is set to equal x_(s), the slave orientational positionin Cartesian space. Thereafter at 558, inverse master kinematics isemployed to determine the master joint position θ_(m) in joint spacewhich corresponds to x_(m), the master position in Cartesian space.Finally, the master is then caused to move to θ_(m) by causingappropriate signals to be sent to the motors on the master as indicatedat 560. It will be appreciated that the surgeon will generally releasethe master to enable it to move into an orientation aligned with theslave orientation.

[0178] Referring again to FIG. 14, after the realignment step at 512,the master is reconnected to the slave as indicated at 513. It will beappreciated that the step 513 is the same as that described above withreference to FIG. 19. The master realignment is described in more detailin Application Serial No. 60/116,842.

[0179] Endoscope Movement

[0180] Referring now to FIGS. 11, 16, and 17, repositioning of theendoscope to capture a different view of the surgical site will now bedescribed. As the surgeon may wish to view the surgical site fromanother position, endoscope arm 302 can selectively be caused to varyits position so as to enable the surgical site to be viewed fromdifferent positions and angular orientations. The arm 302 includesappropriately positioned electrical motors controllable from the controlstation 200. The endoscope arm can thus be regarded as a slave and istypically controllable in a control loop similar to that shown in FIG.11. Regarding the endoscope as another slave, cart 300 has three slaves,the robotic arm assemblies 310 and 304, and two masters 210.

[0181] To vary the position of the endoscope, the surgeon activates aninput at the control station 200. The input can be generated from anyappropriate input device, which can include a depressible button, or avoice control system, or the like. Upon such activation, the controlloops between master 210 and slaves 310 one of which is indicated inFIG. 11, are interrupted and the parts of the control loop on bothmaster sides are operatively linked to a dormant control loop portionsimilar to that of the slave in FIG. 11, but which is arranged tocontrol endoscope arm movement. The surgeon can then change the positionof the endoscope to obtain a different view of the surgical site bymeans of manual inputs on the master controls 210. When the endoscopehas been moved to a desired position, control between master and slaveis re-established in accordance with the methods described aboveincluding automatic assessment of left and right hand allocation betweenmasters and slaves as already discussed.

[0182] An exemplary method and system for robotic movement of theendoscope using both of the master controllers is described in moredetail in Application Serial No. 60/111,711, filed on Dec. 8, 1998, andentitled “Image Shifting for a Telerobotic System,” the full disclosureof which is incorporated herein by reference.

[0183] At times, such as when the scope is moved to an alternativeminimally invasive aperture, or when a scope is removed and replaced,the endoscope may be manually positioned. The steps involved inrepositioning the endoscope are indicated by reference numeral 600 inFIG. 16. To do this a suitable input device is activated.

[0184] The suitable input device is typically in the form of adepressible button on the endoscope arm 302. However other methods suchas voice control or the like can be used instead. The button is similarto the button on the arm 312 as described above. The depressing of sucha button is indicated at 602 in FIG. 16 and is labeled “Depress cameraslave clutch button”. Upon activation of the input button the toolslaves and masters are servo locked at the positions they were atimmediately before activation of the input button.

[0185] When the button is depressed, all the joints on the endoscope arm302 are caused to float as indicated at 609. This will now be describedin greater detail with reference to FIG. 17. As soon as the button isdepressed, the position of the endoscope in joint space immediatelybefore depression of the button is recorded as indicated by θ_(d). Whenthe endoscope arm 302 is then moved to a new desired position, itspresent position indicated by θ_(a) in FIG. 17 is compared with θ_(d) at604 to determine joint positional errors or deviations. These errors arepassed to 606. The torques then determined are zeroed at 608 to causethe joints on the endoscope arm to float to enable repositioning.

[0186] It will be appreciated that floating the endoscope arm can alsobe achieved by setting the gains in 606 to zero or continually updatingθ_(d) to watch θ_(a) as to compute a zero error signal at 604. Similarlyone might disable the endoscope arm controller altogether or zero themotor commands.

[0187] It will also be appreciated that the endoscope could be freed tomove in translation while locked in orientation, analogous to theabove-described methods. Furthermore, one could control the orientationto keep the image aligned with horizontal or vertical, that is keep thetop of the image facing upward (for example, so that gravity isconsistently downward in the image shown to the system operator), whilefloating translational degrees of freedom. Again this is analogous tomethods described above, and can be used to disable and/or float aspectsof the endoscope controller in Cartesian space.

[0188] When the endoscope arm is brought into the required position thebutton is released as indicated at 610 in FIG. 17. Thereafter themasters are realigned with the slaves as indicated at 612 and as alreadydescribed with reference to FIG. 18. Thereafter at 614 control betweenmaster and slave is re-established and as already described withreference to FIG. 19.

[0189] Though the above algorithms for repositioning masters, slavesand/or the endoscope arm were described in isolation, they can also beexecuted in parallel, allowing for simultaneous repositioning of anynumber of system components.

[0190] Left-Right Tool Swap

[0191] Referring now to FIG. 20 of the drawings, in which like referencenumerals are used to designate similar parts unless otherwise stated, animage as viewed by the surgeon, and as captured by the endoscope, isgenerally indicated by reference numeral 800.

[0192] During the course of a surgical procedure, the surgeon is oftencontrolling the actions and movements of the end effectors by inputtingmanual movements and actions on the master controls while viewing thecorresponding end effector movements and actions in the image displayedon the viewer. The left hand master control is typically operativelyassociated with the end effector displayed on the left hand side of theimage and the right hand master control is operatively associated withthe end effector displayed on the right hand side of the image.

[0193] As described above, the surgeon may wish to perform an imageshift by moving the viewing end of the endoscope relative to thesurgical site to view the surgical site from a different position orangle. It could happen that during the conducting of the surgicalprocedure, such as subsequent to an image shift, the end effector whichwas on the left of the displayed image is now on the right, andsimilarly the end effector which was on the right of the displayed imageis now on the left. Furthermore, during the course of, e.g., training,or the like, an operator of the minimally invasive system may wish tooperatively associate the two master controls with a single end effectorso as to enhance a training procedure of the system. This inventionprovides a minimally invasive telesurgical system which provides forselectively permitting operative association of any one or more of aplurality of master controls with any one or more of a plurality of endeffectors.

[0194] The image 800 is schematically indicated in FIG. 21 at anenlarged scale. The image 800 indicates the end effectors 102 at theworking ends 110 of two surgical instruments similar to the surgicalinstrument 100 shown in FIG. 6. In the image, the portions of the shafts104 of the surgical instruments extend outwardly from the image onrespectively a right hand side and a left hand side of the image.Referring again to FIG. 20 of the drawings, the master control device210 on the right hand side of the surgeon is operatively associated withthe slave including the medical instrument defining the shaft extendingoutwardly toward the right hand side of the image 800. Similarly, themaster control device 210 on the left hand side of the surgeon isoperatively associated with the slave including the medical instrumentdefining the shaft extending outwardly toward the left hand side of theimage 800. Accordingly, an anthropomorphic or immersive surgicalenvironment is created at the workstation 200 and the surgeonexperiences an atmosphere of directly controlling actions and movementsof the end effectors 102.

[0195] In FIG. 21A, the end effectors are shown to be at differentpositions in the image. However, it is still clear which shaft 104extends outwardly to the left and right of the image. Accordingly, thesame association between the master control devices and the slavesprevails.

[0196] Referring now to FIG. 22 of the drawings, an image shift hastaken place. This can happen, for instance, where the surgeon wishes tochange the orientation of the surgical site as viewed through theviewer. This can be accomplished by causing the endoscope to displaceangularly about its viewing axis. It will be appreciated that theendoscope mounted on the robotic arm 302, as can best be seen in FIG. 5,can be caused to displace angularly about its viewing axis from thecontrol station 200.

[0197] The new image 802 shown in FIG. 22 for the sake of example, wasbrought about by such angular displacement of the endoscope.Accordingly, the image 802 has undergone an angular displacement, asindicated by arrow 804. The shaft of the medical instrument whichextended to the right of the image now extends to the left of the imageand the shaft of the medical instrument which extended to the left ofthe image now extends to the right of the image. If the associationbetween masters and slaves which existed immediately before the imageshift was to prevail, this would severely impede the surgeon's abilityto carry on with the surgical procedure since left hand control would beassociated with right hand actions and movements of the end effector asdisplayed on the viewer, and vice versa.

[0198] To compensate for such a situation, the minimally invasivesurgical system of the invention causes the association between mastersand slaves which prevailed immediately before the image shift, to beinterrupted and then to be switched or swapped automatically. Once thishas taken place, master control on the surgeon's right hand side isassociated with the slave which includes the shaft extending outwardlyto the right of the new image and the master control on his or her lefthand side is associated with the slave defining the medical instrumenthaving the shaft which extends outwardly to the left of the new image.Thus, the anthropomorphic surgical environment is retained at thecontrol station 200.

[0199] Referring now to FIG. 22B of the drawings, the steps involved incausing the association between a master and a slave to be swapped withthe association of another master and slave will now be discussed.

[0200] The first step, indicated by reference numeral 900 in FIG. 22B ofthe drawings, is to determine the positions of the remote centers orfulcrums 349 of the slaves relative to a Cartesian space coordinatesystem having its origin at the viewing end of the endoscope. This stepwill now be described in greater detail and with reference to FIG. 10 ofthe drawings.

[0201] In other words, in the above discussion, and throughout theremainder of the following discussion, the remote centers or fulcrums349 are considered coincident with the port of entry (as is typical inthe preferred embodiment). In other embodiments, however, these pointsmay not coincide (or even exist, for example, when a distal portion oran endoscopic tool is free to pivot above the insertion point, relyingon the tendency of the tool to pivot at this point with no remote centerimposed) in which case all calculations should be based on the locationof the port of entry. It will also be appreciated that the system maydetermine these port locations from sensor information and pre-existingknowledge of the cart 300, set-up joints, and manipulator arms.Alternatively, the locations could be determined by other sensors or byprocessing the image 800 directly to observe and extrapolate the pivotpoints of the displayed tool shafts.

[0202] As described above with reference to FIG. 10, the positions ofeach fulcrum are generally determined relative to the Cartesiancoordinate system 902, optionally using sensors of the set-up joints.This is indicated at step 911 in the method of FIG. 22B after which (atstep 913) a determination is made as to whether or not the (X,Y)positions of each fulcrum are sufficiently spaced apart relative to eachother to permit the minimally invasive surgical system of the inventionto determine a left hand and right hand allocation for the robotic armassemblies or slaves. This step will now be described in greater detail.

[0203] It will be appreciated that the cart or trolley 300 and therobotic arm assemblies 395, 310, and 302 mounted thereon are notmechanically perfect structures. Thus, in computing the (X,Y)coordinates for each fulcrum 349 positional errors can arise due to,e.g., external forces such as gravity, mechanical misalignments,miscalibration and the like. The range of such positional errors whichcan arise is indicated in FIG. 22A. FIG. 22A indicates the x-x and y-yaxes of the coordinate system 902. The circular part of the shaded areain FIG. 22A represents an area corresponding to an error range or marginresulting from such errors as described above. The parts of the shadedarea diverging outwardly along the x-x axis and from the circular partrepresent regions where the positions of the fulcrums are too close tothe x-x axis for an appropriate allocation to be made.

[0204] To determine whether or not the (X,Y) positions of the fulcrums349 fall in the shaded area, a midpoint between the (X,Y) positions istransformed onto the x-x and y-y axis as indicated in FIG. 22A such thatthe midpoint coincides with the origin 904. With reference again to FIG.22B of the drawings, should the positions of the fulcrums 349 falloutside the shaded error region, the next step as indicated by referencenumeral 915 is performed. If not, an alternative method to allocate leftand right position is followed as indicated by the step 917, as furtherdescribed herein below.

[0205] The step 915 involves a selection or allocation of a right handand left hand position to the slaves. Accordingly, the slave definingthe fulcrum to the left of the x-x axis in FIG. 22A is assigned the lefthand position and similarly the slave defining the fulcrum to the rightof the x-x axis is assigned the right hand position.

[0206] When this allocation has been made the step indicated at 919 isperformed. The step at 919 involves making a comparison between theallocated left and right hand positions with a previous left and righthand allocation. Should these allocations be the same, the associationbetween masters and slaves stays as it was as indicated at 921. Shouldthe allocation not be the same, the step indicated at 923 is performed.

[0207] The step 923 involves requesting a swap between the master andslave associations and will now be described with reference to the blockdiagram shown in FIG. 22C.

[0208] When performing a swap the control loops between the masters andslaves are temporarily interrupted as indicated at 925. This will now bedescribed with reference to FIG. 11 of the drawings. It will beappreciated that the control loop 400 indicates a single control loopwhich operatively associates a single master with a single slave. Theslave side of the loop is indicated below the dashed line in FIG. 11 andthe master side of the loop is indicated above the dashed line. It willbe appreciated that a similar control loop is provided for the othermaster and slave pair. The control loop of each master and slave pairare interrupted at the bilateral controller in step 925. Upon suchinterruption the positions of the masters and slaves are locked inposition by means of the respective master and slave joint controllers420, 560 in the case of each master and slave pair.

[0209] Referring again to FIG. 22C, after interruption of the controlloops, the surgeon is then informed that a swap is about to take placeat step 927. This step typically involves causing a message to bedisplayed in the image at the viewer. The message can require that thesurgeon provide an input to acknowledge his or her awareness of the swapto take place. Such an input can be generated in any appropriate mannersuch as upon depression of a button, or by means of voice control, orthe like. When such an input is generated, operative association betweeneach master and its new associated slave is then established at step929. Thus, referring once again to FIG. 11 of the drawings, the masterside of the control system 400 is linked to the slave side of the othercontrol loop and likewise the master side of the other control loop islinked to the slave side of the control loop 400. Once the control loopshave been connected, each master is moved into alignment with its newassociated slave at step 931, as described with reference to FIG. 18.Each master can then be connected with its new associated slave at step933, as described with reference to FIG. 19 of the drawings. Once thesesteps have been performed, operative control between each master and itsnew slave is fully established as indicated at 914 in FIG. 22B.

[0210] Returning now to FIG. 22B of the drawings, and where thepositions of the fulcrums 349 fall within the error margin as indicatedin FIG. 22A as determined at 911 in FIG. 22B, the step indicated at 917will now be described. At 917, an alternative method of determiningpositions of the fulcrums is employed. This step involves determiningthe orientation of the endoscope relative to the cart 300. To determinethe orientation of the endoscope relative to the cart the positionalsensors are employed to determine whether the viewing end of theendoscope is directed toward or away from the cart. Should the end ofthe endoscope be directed away from the cart, the right hand slave isautomatically allocated a right hand position and the left hand slave isautomatically allocated a left hand position at the step 935, thisallocation presuming a direction of view as indicated by arrow K in FIG.4. Should the viewing end of the endoscope be directed toward the cart,the left hand slave is allocated a right hand position and the righthand slave is allocated a left hand position at the step 935. Again,allocation presumes a direction of view as indicated by the arrow K inFIG. 4. This method is based on the presumption that set up joints,indicated by reference numerals 395 in FIG. 9 do not readily cross eachother.

[0211] It will be appreciated that the endoscope arm 302 can selectivelybe caused to vary its position so as to enable the surgical site to beviewed from different positions and angular orientations. The arm 302includes appropriately positioned electrical motors controllable fromthe control station 200. The endoscope arm can thus be regarded as aslave and is typically controllable in a control loop similar to thatshown in FIG. 11.

[0212] Both masters can optionally be operatively associated with asingle slave, e.g., for training purposes. Employing the methodsdescribed above will also enable a surgeon selectively to control anyone or more of these multiple slave arms with only two masters.Furthermore, two control stations can be operatively associated with asingle cart 200. Thus one master of each control station can then beoperatively linked to a single slave arm and each other master controlwith a single other slave arm. This can be advantageous for e.g.,training purposes, or the like.

[0213] Regarding the endoscope as another slave, the minimally invasivesurgical system of the invention accordingly has three slaves, therobotic arm assemblies 310 and 304, and two masters 210. As describedherein, further slave arms may be incorporated as an optional feature.

[0214] It will be appreciated that the allocation steps described abovefor allocating master and slave association are typically automaticallycarried out at the commencement of a surgical procedure after the slaveshave been brought to initial starting positions at the surgical site.Naturally, in addition, or instead, the allocation steps can beinitiated manually when appropriate by activating a suitable input toinitialize the allocation steps. The steps are also automaticallycarried out when either one or both masters are repositioned relative tothe slaves, when either one or both slaves are repositioned relative tothe associated master or masters and when the endoscope is repositioned,as described earlier in this specification. It is to be appreciated thatwhere an input is required in this specification and where appropriatesuch an input can be by way of any suitable input, such as buttons,cursor selection foot pedal toggling, voice control or any othersuitable form of input.

[0215] It will furthermore be appreciated that the determination ofmaster-slave association, which is computed automatically according toFIGS. 22A and 22B, may be specified manually by way of a suitable inputdevice, such as buttons, a foot pedal, voice control, mouse input, orany other suitable form. If the association is specified manually, onlysteps 911 and 916 need be performed to execute the association.

[0216] In a system with more than two masters or more than two roboticarms with associated instruments, master-slave association willpreferably be entered manually. This can be accomplished by interruptingthe current association to allow the master to translate freely asdescribed above with reference to FIGS. 12 and 13, then using thefloating master as a mouse-like pointing device to highlight and/orselect the image of one of the slaves. To complete the process, themaster is locked and the new association activated using steps 919 and923. Any slaves 9 that are not part of an existing association arelocked in joint space using controller 420. The slave location at thetime of disassociation is stored in memory in 420, and compared againstthe sensor signals to provide appropriate feedback torques.

[0217] Similarly, in a system with more masters than slaves, onlymasters selected by appropriate input devices are associated withslaves, while the remainder are locked using a controller such ascontroller 560.

[0218] Robotic Network

[0219] Referring now to FIGS. 1, 23A and 23B, many of the above stepsmay be used to selectively associate any of a plurality of tools withany of a plurality of input devices. Operator O may initiate a toolselection subroutine 910 by actuating a tool selector input, such as bydepressing foot activated button 208 a of workstation 200 (illustratedin FIG. 2). Assuming operator O is initially manipulating tools A and Bwith input devices 210L and 210R using his or her left and right handsLH and RH, respectively, tool selector procedure 910 will be describedwith reference to a change of association so that input device 21 OL isinstead associated with a tool C, here comprising a tissue stabilizer120.

[0220] Once the tool selector subroutine is activated, the operator willgenerally select the desired tools to be actively driven by the roboticsystem. The surgeon here intends to maintain control over Tool B, butwishes to reposition stabilizer 120. Optionally, operator O will selectbetween the left and right input devices for association with the newlyselected tool. Alternatively, the processor may determine theappropriate left/right association based on factors more fully describedin co-pending U.S. Patent Application Serial No. 60/116,891, filed onJan. 22, 1999, and entitled “Dynamic Association Of Master And Slave InA Minimally Invasive Telesurgical System,” (Attorney Docket No.17516-004700) the full disclosure of which is incorporated herein byreference.

[0221] Optionally, operator O may select the desired tools for use bysequentially depressing selector input 208 a, with the processorsequentially indicating selection of, for example, Tools A and B, then Band C, then A and C, and the like. Controller station 200 may indicatewhich tools are selected on display 800, audibly, or the like. Forexample, the image of the selected tools viewable by the surgeon may becolored green to designate active manipulation status, and/or thedeselected tools may be colored red. Preferably, any deselected tools(for example, Tool A) will be maintained in a fixed position per step914. The tools may be held in position using a brake system and/or byproviding appropriate signals to the drive motors of the tool and armactuation system to inhibit movement of the tool. The tool fixation step914 will preferably be initiated before a master input device isdecoupled from the tool, so that no tool moves absent an instructionfrom an associated master. Tool fixation may occur simultaneously withtool selection. The selected master may be allowed to float, step 916,during and/or after tool fixation and tool selection.

[0222] Once the selected master has been allowed to float, the mastermay be moved into alignment with the selected tool as illustrated inFIG. 23A, as was described above with reference to FIG. 18. Often, thiswill occur while the surgeon keeps a hand on the input device, so thatthe drive motors of the master should move the master at a moderate paceand with a moderate force to avoid injury to the surgeon. Master inputdevice 210L may then be coupled to tool C (stabilizer 120 in ourexample) while tool A is held in a fixed position. This allows theoperator to reposition stabilizer 120 against an alternative portion ofcoronary artery CA. The tool selection process may then be repeated tore-associate the masters with tools A and B while tool C remains fixed.This allows the surgeon to control repositioning of stabilizer 120without significantly interrupting anastomosis of the coronary artery CAwith the internal mammary artery IMA.

[0223] A number of alternative specific procedures may be used toimplement the method outlined in FIG. 23B. Optionally, the interface mayallow the operator to manually move the input devices into apparentalignment with the desired tools while the tool selector button isdepressed. In FIG. 23A, the surgeon might manually move master 210L fromalignment with tool B into approximate alignment with tool C. Theprocessor could then determine the tools to be driven based on theposition of the input devices when the button is released, therebyallowing the operator to “grab” the tools of interest. Some or all ofthe tools (Tools A, B, and C) may optionally be maintained in a fixedconfiguration when the operator is moving the master controllers to grabthe tools.

[0224] Allowing an operator to sequentially control more than tworobotic tools using the operator's two hands can provide significantadvantages. For example, referring again to FIG. 1, by allowing operatorO the ability to select in real time and control any one or two tools100 of cart 300 and auxiliary cart 300A, the surgeon will often be ableto act as his or her own assistant.

[0225] In addition to allowing the operator to safely reposition astabilizer 100 against a coronary artery and the underlying beatingheart during beating heart coronary artery bypass grafting, a variety ofalternative procedures would also be facilitated by such capabilities.As another example, in the procedure of gall bladder removal(cholecystectomy), the surgeon will generally want to first to provideexposure of the organ (retraction) to expose the area of interest. Thisgenerally involves guiding a retractor tool (mounted, for example, to afirst manipulator arm) to expose an area of interest. The area ofinterest may be exposed for viewing through an endoscope mounted, forexample, to a second manipulator arm. The surgeon might thereafter wantto use two hands to direct tools in dissecting tissue covering thecystic duct and artery while the retractor remains stationary. One ofthe two tools (which may be mounted on third and fourth manipulatorarms) can be used to stretch the tissue (traction or grasping) while theother tool is used to cut tissue (sharp dissection) to uncover thevessel and duct structures. Hence, the ability to selectively controlfour manipulators from a single console allows the surgeon to controlthe manipulation, retraction/stabilization, and viewing angle of theprocedure, without having to verbally instruct an assistant.

[0226] At any time during the dissection, the surgeon could have thecapability of adjusting the exposed area of the cystic duct by againselectively associating a master input device in his or her left orright hand with the retractor. Once the desired change in exposure isobtained by repositioning the retractor, the surgeon can deselect theretraction tool, and then select and move the endoscope to a moreappropriate viewing angle for work on the newly exposed tissue.Thereafter, the surgeon can again select the grasping and cutting toolsto manipulate the tissues using both hands.

[0227] The ability to control four or more surgical arms also gives thesurgeon the capability of selecting from among alternative tools basedon tool function and/or anatomical constraints. For example, tools A, B,and C may all have end effectors comprising universal graspers. If thesurgeon is afforded a better approach to tissue dissection by using themanipulator arms associated with tools A and B in certain parts of atwo-handed dissection procedure, but would prefer to use tools B and Cfor alternative portions of the two-handed dissection procedure, theoperator is free to switch back and forth between tools A and C usingtool selection subroutine 910. Similarly, if a cauterizing electrodeblade is desired intermittently during a dissection, the operator mayswitch back and forth between tools A and C to dissect, and thencauterize, and then dissect, etc., without having to wait for anassistant to repeatedly swap tools.

[0228] Advantageously, providing a “redundant” manipulator may reducethe need for a laparoscopic surgical assistant who might otherwise becalled on to perform intermittent functions by manually manipulating atool handle extending from an aperture adjacent the manipulator arms.This can help avoid interference between manual tools, personnel, andthe moving manipulator arms, and may have economic advantages bylimiting the number of highly skilled personnel involved in a roboticsurgical procedure. The procedure time may also be decreased by avoidingthe time generally taken for a lead surgeon to verbally direct anassistant.

[0229] Tool Hand-Off

[0230] Many of the steps described above will also be used when“handing-off” control of a tool between two masters in a tool hand-offsubroutine 920, as illustrated in FIG. 24. Tool hand-off is againinitiated by actuating an appropriate input device, such as bydepressing foot pedal 208 b shown in FIG. 2.

[0231] The tool to be transferred will typically be designated, againusing any of a variety of designation input methods or devices. Thetransfer tool may be coupled to any master input device or devices,including an input device of master control station 200, assistantcontrol station 200A, or auxiliary input 12 of auxiliary cart 300A (asillustrated in FIG. 1). Optionally, the input device which will assumecontrol of the designated tool is also selected in designation step 922,although selection between left and right masters may again be left tothe processor, if desired.

[0232] Once the tool and master are designated, the hand-off tool (andany tool previously associated with the designated master) is fixed, andthe designated master is allowed to float. The master is then alignedand connected with the tool as described above.

[0233] Camera Switch and Right Access Robotic CABG

[0234] The following pertains to an exemplary robotic surgery procedurethat may be performed with the foregoing apparatuses and methods.Referring now to FIGS. 1, 25A, and 25B, a single complex minimallyinvasive surgery will often involve interactions with tissues that arebest viewed and directed from different viewing angles. For example, inperforming a Coronary Artery Bypass Grafting (CABG) procedure on patientP, a portion of the internal mammary artery IMA will be harvested fromalong the internal surface of the abdominal wall. The internal mammaryartery IMA can be used to supply blood to coronary artery CA downstreamof an occlusion, often using an end-to-side anastomosis coupling theharvested end of the IMA to an incision in the side of the occludedcoronary artery. To provide appropriate images to Operator O at mastercontrol station 200, the operator may sequentially select imagesprovided by either a first scope 306 a or a second scope 306 b forshowing on display 800 of the workstation. The camera switch procedurecan be understood through a description of an exemplary CABG procedurein which different camera views may be used. Two scopes are shown inFIG. 25A for illustrative purposes only. If only one image is desired,however, the procedure need not employ two endoscopes but instead needonly use one together with various instruments for actually performingthe procedure.

[0235] As seen in FIGS. 1 and 25A, it may generally be beneficial toaccess heart H primarily through a pattern of apertures 930 disposedalong a right side of patient P. Although the heart is primarilydisposed in the left side of the chest cavity, approaching the heartfrom the left side of the chest as is typically done for MIS heartsurgery may limit the amount of working volume available adjacent thetarget coronary tissues. This lack of working volume can complicatethoracoscopic robotic procedures, as the lack of space can make itdifficult to obtain a panoramic view of the heart surrounding thetissues targeted for treatment, to quickly insert and remove tools, andto retract the heart appropriately for multi-vessel cases.

[0236] By inserting the elongate shafts of instruments 100 through theright side of the patient, the apertures will be further away from thetarget anatomy, including the left internal mammary artery (LIMA) andheart H. This approach can allow the camera to be separated from thetarget tissues by a greater distance, such as when a panorama or“big-picture” view is desired, while the resolution of robotic movementmaintains the surgeon's dexterity when the scope and tools extend acrossthe chest to the heart tissues for close-up views and work. Theright-side approach may also increase the speed with which tools can bechanged, as the additional separation between the aperture and the hearthelps to ensure that the heart is not in the way when delivering toolsto harvest the IMA. When performing multi-vessel cases with theright-side approach, the heart can also be repeatedly retracted andrepositioned so as to sequentially expose target regions of the heart tothe significant working volume available. Hence, different coronaryvessels may selectively be present to operator O for bypassing.

[0237] Advantageously, the right-side approach also facilitatesdissection of the left internal mammary artery (LIMA) using a medial tolateral approach. This dissection approach can provide a well-defineddissection plane, can increase the ease with which branches can be seen,and may provide a view that is more familiar to surgeons accustomed totraditional CABG performed via a median sternotomy.

[0238] As can be seen most clearly in FIGS. 1 and 25B, cart 300 supportsfirst and second tools 100 a, 100 b for manipulating tissues (more maybe used but are not shown) and first scope 306 a, while auxiliary cart300A supports second scope 306 b (and/or other manipulator tools, notshown). The arms of cart 300 preferably extend over the patient from thepatient's left side, and the instruments extend through aperture pattern930. The instrument shafts are generally angled to extend radiallyoutwardly from aperture pattern 930 in a “spoked wheel” arrangement tominimize interference between the manipulators. The exemplaryarrangement has scopes 306 a, 306 b extending through apertures definingthe top and bottom (anterior and posterior relative to the patient)positions of aperture pattern 930, while the manipulation tool shaftsdefine left and right (inferior and superior relative to the patient)positions. Second scope 306 b may be positioned through a lower, moredorsal aperture than shown, with the patient optionally being supportedon a table having an edge RE which is recessed adjacent aperture pattern930 to avoid interference between the auxiliary cart manipulator and thetable.

[0239] A robotic right-side approach CABG procedure might be outlined asfollows:

[0240] 1. General anesthesia is initiated.

[0241] 2. Patient is prepped in a basic supine position with a smallroll under the patient's scapula and back.

[0242] 3. Patient is draped so that drapes start at about the posterioraxillary line.

[0243] LIMA Dissection:

[0244] 4. Camera aperture for second scope 306 b is cut in anappropriate innerspace (usually the 5^(th) intercostal space) onapproximately the anterior axillary line. The first camera port may bepositioned more medially for directing anastomosis, and the like. Ifprovided or desired, the camera aperture for the first scope 306 a iscut in an appropriate innerspace (usually the 4^(th) or 5^(th)intercostal space) slightly posterior to the midclavicular line.Obviously, port placement for the endoscopic tools as well as otherportions of this procedure may vary depending upon the anatomy of theparticular patient in question.

[0245] 5. Initiate insufflation at approximately 10 mm Hg.

[0246] 6. Manipulation tool apertures are cut as appropriate (usually inthe 3^(rd) and 6^(th) or 7^(th) intercostal spaces for manipulationinstruments 100 a, 100 b) a few centimeters medial to the anteriorauxiliary line. Additional tool ports are placed as desired.

[0247] 7. Robotic instruments 100 a, 100 b, 306 a, 306 b introducedthrough apertures and robotic telesurgical control system is initiated.

[0248] 8. LIMA harvesting is initiated by locating midline to establishthe beginning of dissection. Harvesting may be viewed and directed usingsecond scope 306 b.

[0249] 9. LIMA is located by moving laterally using blunt dissection andcautery as desired.

[0250] 10. Left pleura need not be entered, although insufflation canhelp keep left lung out of the way.

[0251] RIMA Dissection (if desired)

[0252] 11. The right internal mammary artery (RIMA) may be harvestedusing steps similar to 1-10 above, optionally through apertures disposedalong the left side of the patient's chest.

[0253] Pericardiotomy

[0254] 12. Incision may be made at surgeon's discretion. Preferably, anyincision will be high up on the pericardium, so that an attachment ofthe mediastinum to the chest forms a tent to enhance exposure of theheart.

[0255] IMA Preparation

[0256] 13. IMA(s) may be prepared per surgeon's preference, typicallywhile still attached to the chest.

[0257] Aorta Exposure

[0258] 14. Aorta is exposed by extending the pericardiotomy cephalad asdesired. The pulmonary artery and any other adhesions may be dissectedoff so that the aorta can be clamped for cardio-pulmonary bypass and/orproximal grafts. As can be understood with reference to the descriptionabove of FIG. 23A, cardioplegia may be avoided by using a manual orrobotic cardiac tissue stabilizer mounted to, for example, auxiliarycart 300A during the anastomosis.

[0259] Coronary Artery Exposure

[0260] 15. Target coronary artery or arteries can be exposed to workingvolume by retracting and repositioning heart as desired, and standardtechniques may be used to expose and incise the exposed coronaryarteries.

[0261] Anastomosis

[0262] 16. Anastomosis may be performed using needle-grasping tools 100a, 100 b while viewing display 800, as illustrated in FIG. 23A.

[0263] Suturing and exposure of the aorta and coronary artery orarteries may at least in part be performed while viewing the moreanterior-to-posterior field of view provided from first scope 306 a, asmay portions of all other steps throughout the CABG procedure. When thesurgeon desires to change views between first and second image capturedevices, the surgeon may initiate the view change procedure byactivating a view change input device, possibly in the form of yetanother foot switch. The tissue manipulation tools will be briefly fixedin position, and the display will shift between the image capturedevices—for example, from the image provided from first scope 306 a, tothe image provided from second scope 306 b.

[0264] Optionally, the processor can reconfigure the coordinatetransformations between the masters and the end effectors when changingbetween two different image capture devices to re-establish an at leastsubstantially connected relationship. This transformation modificationis similar to the process described above for a change in scopeposition, but will generally also accommodate the differences in supportstructure of the image capture devices. In other words, for example, themaster and/or slave kinematics 408, 412 (see FIG. 11) may be redefinedto maintain a correlation between a direction of movement of the inputdevice 210 and a direction of movement of an image of the end effector102 as shown in display 202 when viewing the end effector from adifferent scope. Similarly, when moving second scope 306 b (supported byauxiliary cart 300A) as a slave after a scope change from scope 306 a(which is supported by cart 300), the slave kinematics 412, slaveinput/output 414, and slave manipulator geometry 416 may all bedifferent, so that the control logic between the master and slave may berevised as appropriate.

[0265] More easily implemented approaches might allow the operator O toswitch views between scopes 306 a and 306 b without major softwarerevisions. Using software developed to perform telesurgery with a singlemaster control station 200 coupled to a single three arm cart 300 (seeFIG. 1), switching the view to scope 306 b from scope 306 a might beaccomplished while maintaining the substantially connected relationshipby “fooling” the processor of the master control station into believingthat it is still viewing the surgery through scope 306 a. Moreaccurately, the processor may be fed signals which indicate that themiddle set-up joint 395 and/or manipulator arm 302 of cart 300 aresupporting scope 306 a at the actual orientation of scope 306 b. Thismay be accomplished by decoupling the position sensing circuitry of themiddle set-up joint and/or manipulator of cart 300 from the processor,and instead coupling an alternative circuit that transmits the desiredsignals. The alternative “fooling” circuit may optionally be in the formof a sensor system of an alternative set-up joint and/or manipulator302, which might be manually configured to hold a scope at theorientation of scope 306 b relative to cart 300, but which need notactually support anything. The image may then be taken from scope 306 bsupported by auxiliary cart 300A, while the slave position signals x_(s)(See FIG. (11) are taken from the alternative set-up joint. As describedabove, so long as the orientation of the end effectors relative to thescope are accurately known, the system can easily accommodate positionalcorrections (such as by the translational clutching procedure describedabove).

[0266] Alternative telesurgical networks are schematically illustratedin FIGS. 26 and 27. As mentioned above, an operator O and an AssistantA3 may cooperate to perform an operation by passing control ofinstruments between input devices, and/or by each manipulating their owninstrument or instruments during at least a portion of the surgicalprocedure. Referring, now to FIG. 26, during at least a portion of asurgical procedure, for example, cart 305 is controlled by Operator Oand supports an endoscope and two surgical instruments. Simultaneously,for example, cart 308 might have a stabilizer and two other surgicalinstruments, or an instrument and another endoscope A3. The surgeon oroperator O and assistant A3 cooperate to perform a stabilized beatingheart CABG procedure by, for example, passing a needle or other objectback and forth between the surgical instruments of carts 305, 308 duringsuturing, or by having the instruments of cart 308 holding the tissue ofthe two vessels being anastomosed while the two instruments of cart 305are used to perform the actual suturing. Such cooperation heretofore hasbeen difficult because of the volumetric space required for human handsto operate. Since robotic surgical end effectors require much less spacein which to operate, such intimate cooperation during a delicatesurgical procedure in a confined surgical space is now possible.Optionally, control of the tools may be transferred or shared during analternative portion of the procedure.

[0267] Referring now to both FIGS. 26 and 27, cooperation betweenmultiple systems is also possible. The choice of how many masters andhow many corresponding slaves to enable on a cooperating surgical systemis somewhat arbitrary. Within the scope of the present invention, onemay construct a single telesurgical system's architecture to handle fiveor six manipulators (e.g., two masters and three or four slaves) or tenor twelve manipulators (e.g., four masters and six or eightmanipulators), although any number is possible. For a system havingmultiple master controls, the system may be arranged so that twooperators can operate the same surgical system at the same time bycontrolling different slave manipulators and swapping manipulators aspreviously described.

[0268] Alternatively, it may be desirable to have a somewhat modulartelesurgical system that is capable both of conducting one particularsurgical operation with only one operator and, for example, five or sixmanipulators, and which is also capable of coupling to another modularsystem having five or six manipulators to perform a second surgicalprocedure in cooperation with a second operator driving the secondsystem. For such modular systems, five or six manipulator arms arepreferably supported by the architecture, although any number may beincorporated into each system. One advantage of the modular system overa single, larger system is that when decoupled, the modular systems maybe used for two separate simultaneous operations at two differentlocations, such as in adjacent operating rooms, whereas such might bequite difficult with a single complex telesurgical system.

[0269] As can be understood with reference to FIG. 26, a simple mannerof having two surgical systems, each having an operator, to cooperateduring a surgical procedure is to have a single image capture device,such as an endoscope, produce the image for both operators. The imagecan be shared with both displays by using a simple image splitter. Ifimmersive display is desired, the two systems might additionally share acommon point of reference, such as the distal tip of the endoscope, fromwhich to calculate all positional movements of the slave manipulators,all as previously described in U.S. Appl. No. 60/128,160. With theexception of the imaging system, each control station might beindependent of the other, and might be operatively coupled independentlyto its associated tissue manipulation tools. Under such a simplecooperative arrangement, no swapping of slave manipulators from onesystem to another would be provided, and each operator would havecontrol over only the particular slave manipulators attached directly tohis system. However, the two operators would be able to pass certainobjects back and forth between manipulators, such as a needle during ananastomosis procedure. Such cooperation may increase the speed of suchprocedures once the operators establish a rhythm of cooperation. Such anarrangement scenario may, for example, be used to conduct a typical CABGprocedure, such that one operator would control the endoscope and twotissue manipulators, and the other operator would control two or threemanipulators to aid in harvesting the IMA and suturing the arterialblood source to the blocked artery downstream of the particular blockedartery in question. Another example where this might be useful would beduring beating heart surgery, such that the second operator couldcontrol a stabilizer tool in addition to two other manipulators andcould control the stabilizer while the first operator performed ananastomosis.

[0270] One complication of simple cooperative arrangements is that ifthe first operator desired to move the image capture device, themovement might alter the image of the surgical field sufficiently thatthe second operator would no longer be able to view his slavemanipulators. Thus, some cooperation between the operators, such asaudible communications, might be employed before such a maneuver.

[0271] A slightly more complicated arrangement of surgical manipulatorson two systems within the scope of the present invention, occurs whenoperators are provided with the ability to “swap” control of manipulatorarms. For example, the first operator is able to procure control over amanipulator arm that is directly connected to the second operator'ssystem. Such an arrangement is depicted in FIG. 26.

[0272] With the ability to operatively hook multiple telesurgicalsystems together, an arrangement akin to a surgical production line canbe envisioned. For example, a preferred embodiment of the presentinvention is shown in FIG. 27. Therein, a single master surgeon Ooccupies a central master control operating room. Satellite operatingrooms (ORs) 952, 954 and 956 are each operatively connected to thecentral master console via switching assembly 958, which is selectivelycontrolled by Operator O. While operating on a first patient P1 in OR956, the patients in ORs 954 and 952 are being prepared by assistants A2and A3, respectively. During the procedure on patient P1, patient P3becomes fully prepared for surgery, and A3 begins the surgery on themaster control console dedicated to OR 952 by controlling manipulatorassembly 964. After concluding the operation in OR 956, Operator Ochecks with A3 by inquiring over an audio communications network betweenthe ORs whether A3 requires assistance. OR 950 might additionally have abank of video monitors showing the level of activity in each of the Ors,thereby permitting the master surgeon to determine when it would be bestto begin to participate in the various ongoing surgeries, or to handcontrol off to others to continue or complete some of the surgeries.

[0273] Returning to the example, if A3 requests assistance, O selects OR952 via switching assembly 958, selects a cooperative surgery set-up onan OR-dedicated switching assembly 960, and begins to controlmanipulator assembly 962. After completion of the most difficult part ofthe surgery in OR 952, O switches over to OR 954, where patient P2 isnow ready for surgery.

[0274] The preceding description is a mere example of the possibilitiesoffered by the cooperative coupling of masters and slaves and varioustelesurgical systems and networks. Other arrangements will be apparentto one of skill in the art reading this disclosure. For example,multiple master control rooms can be imagined in which several mastersurgeons pass various patients back and forth depending on theparticular part of a procedure being performed. The advantages ofperforming surgery in this manner are myriad. For example, the mastersurgeon 0 does not have to scrub in and out of every procedure. Further,the master surgeon may become extremely specialized in performing partof a surgical procedure, e.g., harvesting an IMA, by performing justthat part of a procedure over and over on many more patients than heotherwise would be able to treat. Thus, particular surgical procedureshaving distinct portions might be performed much more quickly by havingmultiple surgeons, with each surgeon each performing one part of theprocedure and then moving onto another procedure, without scrubbingbetween procedures. Moreover, if one or more patients (for whateverreason) would benefit by having a surgeon actually be present, analternative surgeon (different from the master surgeon) may be on callto one or more operating rooms, ready to jump in and address thepatient's needs in person, while the master surgeon moves on treatanother patient. Due to increased specialization, further advances inthe quality of medical care may be achieved.

[0275] In addition to enabling cooperative surgery between two or moresurgeons, operatively hooking two or more operator control stationstogether in a telesurgical networking system also may be useful forsurgical training. A first useful feature for training students orsurgeons how to perform surgical procedures would take advantage of a“playback” system for the student to learn from a previous operation.For example, while performing a surgical procedure of interest, asurgeon would record all of the video information and all of the dataconcerning manipulation of the master controls on a tangible machinereadable media. Appropriate recording media are known in the art, andinclude videocassette or Digital Video Disk (DVD) for the video imagesand/or control data, and Compact Disk (CD), e.g., for the servo datarepresenting the various movements of the master controls.

[0276] If two separate media are used to record the images and the servodata, then some method of synchronizing the two would be desirableduring feedback, to ensure that the master control movementssubstantially mirror the movements of the slave manipulators in thevideo image. A crude but workable method of synchronization mightinclude a simple time stamp and a watch. Preferably, both video imagesand servo data would be recorded simultaneously on the same recordingmedium, so that playback would be automatically synchronized.

[0277] During playback of the operation, a student could place his handson the master controls and “experience” the surgery, without actuallyperforming any surgical manipulations, by having his hands guided by themaster controls through the motions of the slave manipulators shown onthe video display. Such playback might be useful, for example, inteaching a student repetitive motions, such as during suturing. In sucha situation, the student would experience over and over how the mastersmight be moved to move the slaves in such a way as to tie sutures, andthus hopefully would learn how better to drive the telesurgical systembefore having to perform an operation.

[0278] The principles behind this playback feature can be built upon byusing a live hand of a second operator instead of simple data playback.For example, two master control consoles may be connected together insuch a way that both masters are assigned to a single set of surgicalinstruments. The master controls at the subordinate console would followor map the movements of the masters at the primary console, but wouldpreferably have no ability to control any of the instruments or toinfluence the masters at the primary console. Thus, the student seatedat the subordinate console again could “experience” a live surgery byviewing the same image as the surgeon and experiencing how the mastercontrols are moved to achieve desired manipulation of the slaves.

[0279] An advanced version of this training configuration includesoperatively coupling two master consoles into the same set of surgicalinstruments. Whereas in the simpler version, one console was subordinateto the other at all times, this advanced version permits both mastercontrols to control motion of the manipulators, although only one couldcontrol movement at any one time. For example, if the student werelearning to drive the system during a real surgical procedure, theinstructor at the second console could view the surgery and follow themaster movements in a subordinate role. However, if the instructordesired to wrest control from the student, e.g., when the instructordetected that the student was about to make a mistake, the instructorwould be able to override the student operator by taking control overthe surgical manipulators being controlled by the student operator. Theability to so interact would be useful for a surgeon supervising astudent or second surgeon learning a particular operation. Since themasters on the instructor's console were following the surgery as if hewere performing it, wresting control is a simple matter of clutchinginto the surgery and overriding the control information from the studentconsole. Once the instructor surgeon had addressed the issue, either byshowing the student how to perform a certain part of the surgicalprocedure or by performing it himself, the instructor could clutch outof the operation and permit the student to continue.

[0280] An alternative to this “on-off” clutching—whereby the instructorsurgeon is either subordinate to the student or in command—would be avariable clutch arrangement. For example, again the instructor issubordinate to the student's performance of a procedure, and has hismasters follow the movement of the student's master controls. When theinstructor desires to participate in the procedure, but does not desireto wrest all control from the student, the instructor could begin toexert some control over the procedure by partially clutching and guidingthe student through a certain step. If the partial control wasinsufficient to achieve the instructor's desired result, the instructorcould then completely clutch in and demonstrate the desired move, asabove. Variable clutching could be achieved by adjusting an inputdevice, such as a dial or a foot pedal having a number of discretesettings corresponding to the percentage of control desired by theinstructor. When the instructor desires some control, he or she couldoperate the input device to achieve a setting of, for example, 50percent control, in order to begin to guide the student's movements.Software could be used to calculate the movements of the end effectorsbased on the desired proportionate influence of the instructor'smovements over the student's. In the case of 50% control, for example,the software would average the movements of the two sets of mastercontrols and then move the end effectors accordingly, producingresistance to the student's desired movement, thereby causing thestudent to realize his error. As the surgeon desires more control, he orshe could ratchet the input device to a higher percentage of control,finally taking complete control as desired.

[0281] Other examples of hooking multiple telesurgical control stationstogether for training purposes will be apparent to one of skill in theart upon reading this disclosure. Although these training scenarios aredescribed by referring to real surgery, either recorded or live, thesame scenarios could be performed in a virtual surgical environment, inwhich, instead of manipulating the tissue of a patient (human or animal)cadaver, or model, the slave manipulators could be immersed, in avirtual sense, in simulation software. The software would then create asimulated virtual surgical operation in which the instructor and/orstudent could practice without the need for a live patient or anexpensive model or cadaver.

[0282] While the present invention has been described in some detail, byway of example and for clarity of understanding, a variety of changes,adaptation, and modifications will be obvious to those of skill in theart. For example and without limiting effect, robotic systems havingmore than four manipulators and/or more that two scopes may be provided.The manipulator arms can all be mounted to a single support base, ormight be arranged with two arms on each of two separate support bases.Hence, the scope of the present invention is limited solely by theappended claims.

What is claimed is:
 1. A robotic surgical system comprising: an imagingsystem transmitting an image from an image capture device to a display;first, second, and third manipulator arms, each arm supporting anassociated surgical instrument; a master controller disposed adjacentthe display, the master having a first input device manipulatable by afirst hand of an operator, and a second input device manipulatable by asecond hand of the operator; and a processor operatively coupling eachinput device of the master to an associated arm so that movement of theinput device effects movement of the associated surgical instrument. 2.The robotic surgical system of claim 1, wherein the surgical instrumentassociated with the third arm comprises another image capture device. 3.The robotic surgical system of claim 2, wherein the processor transmitsarm movement signals to the arm in response to input commands from theinput devices according to a first transformation when the display showsthe image from the image capture device, and according to a secondtransformation when the display shows the image from the other imagecapture device, the processor deriving the first and secondtransformations so that images of the surgical instruments appearsubstantially connected to the input devices when the input devicesmove.
 4. The robotic surgical system of claim 1, wherein the third armhas a stationary configuration, the third arm inhibiting movement of theassociated surgical instrument at the surgical site when the inputdevices move and the arm is in the stationary configuration.
 5. Therobotic surgical system of claim 4, wherein the surgical instrumentassociated with the third arm comprises a surgical tool selected fromthe group consisting of a stabilizer and a retractor.
 6. The roboticsurgical system of claim 5, wherein the surgical instrument associatedwith the third arm comprises a coronary tissue stabilizer for beatingheart surgery.
 7. The robotic surgical system of claim 4, wherein thethird arm comprises a linkage having a plurality of joints and a brakesystem coupled to the joints, the brake system inhibiting articulationof the joints.
 8. The robotic surgical system of claim 7, wherein thethird arm has a passive configuration, the brake system allowing manualarticulation of the arm in response to a passive configuration signal.9. The robotic surgical system of claim 8, wherein the third armcomprises a passive manipulator, having a plurality of passive jointswithout actuators drivingly engaging the joints.
 10. The roboticsurgical system of claim 4, wherein the third arm further has a drivenconfiguration, the third arm comprising a linkage with a plurality ofjoints and a plurality of actuators drivingly engaging the linkage toeffect articulation of the joints so that the third arm moves theassociated surgical instrument at the surgical site when the third armis in the driven configuration.
 11. The robotic surgical system of claim1, wherein the imaging system comprises a fourth manipulator armsupporting the image capture device, the fourth manipulator armcomprising a linkage having a plurality of joints and a plurality ofactuators drivingly engaging the linkage to effect articulation of thejoints.
 12. The robotic surgical system of claim 11, wherein the secondand third arms each comprise a linkage with a plurality of joints and aplurality of actuators drivingly engaging the linkage to effectarticulation of the joints, the processor coupling the master to thearms so that: the first arm can move the associated surgical instrumentat the surgical site in response to movement of the first input device;and the second arm can move the associated surgical instrument inresponse to movement of the second input device.
 13. The roboticsurgical system of claim 12, further comprising at least one armselector input coupled to the processor, the processor selectivelydecoupling the first arm from the first input device and operativelycoupling the first input device with at least one member of the groupconsisting of the second arm, the third arm, and the fourth arm inresponse to an arm selection signal from the arm selector, the coupledarm moving in response to movement of the first input device.
 14. Therobotic surgical system of claim 13, wherein the processor selectivelydecouples the second arm from the second input device and operativelycouples the second input device with at least one member of the groupconsisting of the first arm, the third arm, and the fourth arm inresponse to a signal from the arm selector, the coupled arm moving inresponse to movement of the second input device.
 15. The roboticsurgical system of claim 12, wherein the surgical instruments associatedwith the first and second arms comprise first and second end effectorsfor treating or manipulating tissue.
 16. The robotic surgical system ofclaim 12, wherein the arms remain in a stationary configuration when notoperatively associated with an input device, and when the operatorperforms surgical tasks with the coupled arms by manipulating the firstand second input devices.
 17. The robotic surgical system of claim 11,wherein the image capture device comprises a fourth surgical instrument,wherein each surgical instrument comprises an endoscopic instrumenthaving an elongate shaft with a proximal end adjacent the manipulatorand a distal end insertable into a patient through a minimally invasivesurgical aperture.
 18. The robotic surgical system of claim 17, whereinthe manipulators support the surgical instruments so that the shaftsextend radially outwardly from a pattern of the apertures.
 19. Therobotic surgical system of claim 18, wherein two of the surgicalinstruments comprising endoscopes, and wherein two of the surgicalinstruments comprise endoscopic surgical tools for manipulating ortreating tissue, the manipulators supporting the instruments so that theendoscope apertures define a top and bottom of the pattern and the toolapertures define left and right sides of the pattern.
 20. The roboticsurgical system of claim 1, further comprising an assistant inputdevice, wherein the processor can selectably associate one of the armswith the assistant input device, or with an input device of the master,so that the one arm moves in response to movement of the selected inputdevice.
 21. The robotic surgical system of claim 20, wherein theprocessor can selectable couple any input included in the groupcontaining the first input device, the second input device, and theassistant input device, with any arm included in the group containingthe first arm and the second arm so that the selected arm moves inresponse to manual movement of the selected input.
 22. The roboticsurgical system of claim 20, wherein the processor moves the surgicalinstrument associated with one arm relative to the image capture deviceso that an image of the associated surgical instrument as shown in thedisplay appears substantially connected with the selected input deviceof the master.
 23. The robotic surgical system of claim 22, wherein theprocessor moves the selected input device into alignment with theinstrument associated with the selected arm when the arm is selected.24. The robotic surgical system of claim 22, further comprising anassistant display visible from adjacent the arms so that the assistantcan view the surgical site while removing and replacing the surgicalinstruments.
 25. The robotic surgical system of claim 22, furthercomprising an assistant controller station including the assistant inputdevice and an assistant display, wherein the processor moves a surgicalinstrument associated with the selected arm relative to the imagecapture device so that an image of the surgical instrument as shown inthe assistant display appears substantially connected with the assistantinput device.
 26. A robotic surgical system comprising: a plurality ofmanipulator arms; a plurality of surgical instruments, each instrumentmounted to an associated arm; a master controller station having amaster display for viewing by an operator, a first input device formanipulation by a first hand of the operator, and a second input devicefor manipulation by a second hand of the operator; an assistantcontroller having an assistant input device for manipulation by a handof an assistant; and a processor selectably operatively coupling thecontrollers to the arms to effect movement of the surgical instrumentsin response to movement of the input devices.
 27. The robotic surgicalsystem of claim 26, further comprising an arm selector coupled to theprocessor for selecting between a plurality of modes, wherein a firstarm is operatively associated with the assistant controller when theprocessor is in a first mode, and wherein the first arm is operativelyassociated with the master controller when the controller is in a secondmode, the operatively associated arm moving the associated surgicalinstrument in response to movement of the operatively associated inputdevice.
 28. The robotic surgical system of claim 26, wherein a secondarm is operatively associated with the master controller when theprocessor is in the first mode, and wherein the second arm isoperatively associated with the assistant controller when the processoris in the second mode.
 29. A robotic surgical method comprising:robotically moving first and second surgical instruments at a surgicalsite with first and second robotic manipulator arms by manipulatingfirst and second input devices with first and second hands of anoperator, respectively; selectively associating the first input devicewith a third manipulator arm; robotically moving a third surgicalinstruments at the surgical site with the third manipulator arm bymanipulating the first input device with the first hand of the operator.30. A robotic surgical method comprising: robotically moving first andsecond surgical instruments at a surgical site with first and secondrobotic manipulator arms by manipulating first and second input deviceswith first and second hands of an operator, respectively; positioning athird surgical instrument at the surgical site by articulating a linkageof a third manipulator; impeding movement of the positioned thirdsurgical instrument at the surgical site by inhibiting movement of thethird manipulator.
 31. A robotic surgical method comprising: roboticallypositioning a surgical instruments at a surgical site with a manipulatorarm by manipulating a first input device with a hand of a firstoperator; selectively associating the manipulator arm with a secondinput device; robotically moving the surgical instrument at the surgicalsite with the manipulator arm by manipulating the second input devicewith a hand of a second operator.
 32. A robotic surgical methodcomprising: showing on a display a first view of a surgical site from afirst image capture device; robotically moving a surgical instrument atthe surgical site with a manipulator arm by manipulating an input devicewith a hand of an operator while the operator views the first view ofthe surgical site on the display; selectively associating a second imagecapture device with the display; and robotically moving the surgicalinstrument at the surgical site with the arm by manipulating the inputdevice while the operator views a second view of the surgical site fromthe second image capture device on the display.
 33. A robotic cardiacsurgery method comprising: introducing an image capture device into achest cavity of a patient through an aperture disposed along a rightside of the patient; displaying an image of a surgical site adjacent theheart from the image capture device to an operator; performing asurgical procedure on the heart by moving a surgical instrument at thesurgical site with at least one robotic manipulator arm while thesurgical instrument extends through another aperture disposed along theright side of the patent.
 34. The robotic surgical system of claim 6,wherein the stabilizer comprises two substantially aligned tissueengaging surfaces having ridges, each tissue engaging surface having anopening therethrough, the tissue engaging surfaces movable relative toeach other substantially in-plane to define a stabilized tissue accessopening therebetween.
 35. A robotic surgical system comprising: a firstinput device manipulatable by a hand of an operator; a first robotic armassembly including a first manipulator arm for moving a first surgicalinstrument; a second robotic arm assembly including a second manipulatorarm for moving a second surgical instrument; and a control systemcoupling the first input device to the first and second robotic armassemblies, the control system permitting selective operativeassociation of the first input device with the first robotic armassembly and permitting selective operative association of the firstinput device with the second robotic arm assembly.
 36. The roboticsurgical system of claim 35, wherein the control system has a pluralityof selectable modes, the control system in a first mode operativelyassociating the first robotic arm assembly with the first input deviceso that manipulation of the first input device effects movement of thefirst surgical instrument, the control system in a second modeoperatively associating the second robotic arm assembly with the firstinput device so that manipulation of the first input device effectsmovement of the second surgical instrument.
 37. The robotic surgicalsystem of claim 36, wherein the control system is configured so thatmanipulation of the first input device when the control system is in thefirst configuration does not effect movement of the second surgicalinstrument.
 38. The robotic surgical system of claim 36, furthercomprising a second input device, the control system in the first modeoperatively associating the second robotic arm assembly with the secondinput device so that manipulation of the second input device effectsmovement of the second surgical instrument, the control system in thesecond mode operatively associating the first robotic arm assembly withthe second input device so that manipulation of the second input deviceeffects movement of the first surgical instrument.
 39. The roboticsurgical system of claim 38, wherein each surgical instrument comprisesa surgical end effector, and further comprising an image capture devicecoupled to a display adjacent to the first input device to show an imageof at least one of the surgical end effectors to the operator.
 40. Therobotic surgical system of claim 39, wherein the control system isconfigured to select between the first and second configurations inresponse to locations of the first and second surgical instrumentsrelative to the image capture device.
 41. The robotic surgical system ofclaim 40, further comprising an image capture assembly comprising animage capture arm movably supporting the image capture device, thecontrol system configured to change between the first configuration andthe second configuration in response to movement of the image capturedevice.
 42. The robotic surgical system of claim 41, wherein the imagecapture device comprises an endoscope, the first and second instrumentscomprise minimally invasive tools having elongate shafts suitable forinsertion to an internal surgical site via a minimally invasive surgicalaperture, the image capture arm comprising a robotic arm assemblycoupled to the control system so that the control system can effectmovement of the endoscope per endoscope commands from the operator. 43.The robotic surgical system of claim 39, further comprising a tool swapinput device coupled to the control system, the control system changingbetween the first configuration and the second configuration in responseto input to the tool swap input device by the operator.
 44. The roboticsurgical system of claim 35, wherein the control system can inhibitmovement of the first instrument.
 45. The robotic surgical system ofclaim 35, wherein the control system is configured to effect movement ofthe first input device in response to a change of operative associationof the first input device between the first robotic arm assembly and thesecond robotic arm assembly.
 46. The robotic surgical system of claim45, wherein the master input device comprises a linkage having aplurality of joints and a plurality of motors drivingly engaging thejoints to move a handle of the first master input device with aplurality of degrees of freedom, and wherein the control system isconfigured to effect movement of the handle in at least one of theplurality of degrees of freedom in response to the change of operativeassociation.
 47. The robotic surgical system of claim 46, wherein thecontrol system is configured to effect reorientation of the handle inresponse to the change of operative association so that the input deviceappears at least substantially connected with an image of theoperatively associated surgical instrument.
 48. The robotic surgicalsystem of claim 47, wherein the control system is configured to inhibitmovement of the operatively associated surgical instrument while thefirst master controller device moves.
 49. The robotic surgical system ofclaim 35, wherein the first surgical instrument comprises an endoscope,and wherein the second surgical instrument comprises a surgical toolhaving a surgical end effector adapted for treating tissue.
 50. Arobotic surgical system comprising: two input devices; an endoscoperobotic arm assembly; at least two medical instrument manipulator armassemblies; and a control system permitting selection of which one ofthe input devices is to be operatively associated with which one of themanipulator assemblies.
 51. A robotic surgical system comprising: aplurality of input devices; a plurality of manipulator arms, eachmanipulator arm having an instrument holder; a plurality of surgicalinstruments mountable to the instrument holders, the plurality ofsurgical instruments including an image capture device and a tool havinga surgical end effector for treating tissue; and a control systemcoupling the input devices with the manipulator arms, the control systemselectably associating each input device with a manipulator arm.
 52. Therobotic surgical system of claim 51, the image capture device isalternatively mountable to first and second instrument holders of firstand second manipulator arms, respectively, and wherein the controlsystem can reconfigure operative association between the input devicesand the manipulator arms when the tool is removed from the firstinstrument holder and mounted to the second instrument holder.
 53. Therobotic surgical system of claim 51, wherein the control system includesan input/manipulator association memory storing an associationconfiguration identifying one or more coupling pairs, each coupling pairincluding at least one input device and at least one manipulator arm,the coupling pairs alterable by a signal to identify coupling betweenthe at least one input device and at least one alternative manipulatorarm.
 54. The robotic surgical system of claim 53, wherein the controlsystem comprises a pre-processor gathering positional informationregarding the input devices and the manipulator arms, a controllergenerating comparison signals in response to a comparison between thepositional information for each input device with the positionalinformation for an associated manipulator according to the associationconfiguration, and a post-processor transmitting commands to themanipulator arms in response to the comparison signals.
 55. The roboticsurgical system of claim 54, wherein the controller compares thepositional information and the post-processor transmits commands to themanipulator arms more than about 900 times per second.
 56. A minimallyinvasive robotic surgical system comprising: two input devices; at leasttwo medical instrument robotic arm assemblies, one of the input devicesbeing operatively associated with one of the robotic arm assemblies tocause movement of the robotic arm assembly in response to inputs on theinput device, and the other input device being operatively associatedwith another of the robotic arm assemblies to cause movement of thatother robotic arm assembly in response to inputs on that other inputdevice; and a control system coupling the input devices with the roboticarm assemblies, the control system enabling selective swapping so as tocause the input device to be operatively associated with the robotic armassembly which was operatively associated with the other input device,and to cause the other input device to be operatively associated withthe robotic arm assembly which was operatively associated with the inputdevice.
 57. A minimally invasive robotic surgical system comprising: twoinput devices; an image capture robotic arm assembly; at least twomedical instrument robotic arm assemblies, one of the input devicesbeing operatively associated with one of the instrument robotic armassemblies to cause movement of the instrument robotic arm assembly inresponse to inputs on the input device and the other input device beingoperatively associated with another of the instrument robotic armassemblies to cause movement of that other instrument robotic armassembly in response to inputs on that other input device; and a controlsystem enabling selective operative association between at least one ofthe input devices and robotic arm assembly to permit the position of animage capture device to be changed using the at least one input device.58. A robotic surgical method comprising: robotically moving a firstsurgical instrument using a first manipulator arm by manipulating afirst input device with a hand; reconfiguring a control system byentering a command, the control system coupling the first input devicewith the first manipulator arm and with a second manipulator arm; androbotically moving a second surgical instrument using the secondmanipulator arm by manipulating the input device with the hand after thereconfiguring step.
 59. A robotic surgical method comprising:robotically moving a surgical instrument using a manipulator arm bymanipulating a first input device with a first hand; reconfiguring acontrol system by entering a command, the control system coupling thefirst input device and a second input device with the manipulator arm;and robotically moving the surgical instrument using the manipulator armby manipulating the second input device with a second hand after thereconfiguring step.